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FOREWORD 


How  does  a  Federal  agency  locate  a  personal  record  in  one  of  its 
massive  files  with  only  a  limited  amount  of  information  for  guidance? 
The  Privacy  Act  of  1974  presented  numerous  agencies  with  just  this 
problem  when  it  specified:    "It  shall  be  unlawful  for  any  Federal, 
state,  or  local  government  agency  to  deny  to  any  individual  any 
right,  benefit,  or  privilege  provided  by  law  because  of  such 
individual's  refusal  to  disclose  his  Social  Security  account  number." 

Those  Federal  agencies  denied  the  use  of  Social  Security  numbers  are 
not  the  only  ones  interested  in  the  solution  to  this  problem.  For 
instance,  one  might  be  required  to  locate  individuals  in  a  file 
who  have  been  inadvertently  or  illegally  recorded  under  more  than 
one  account  number.    Similarly,  the  results  of  a  study  of 
methodologies  for  retrieving  an  individual's  record  without  the 
use  of  a  universal  identifier  can  be  used  to  determine  the  threats 
to  an  individual's  privacy. 

This  report  was  written  in  response  to  this  growing  need  for 
retrieving  information  using  non-unique  identifiers.    Presented  are 
selected  methodologies  for  assisting  Federal  agencies  in  selecting 
retrieval  algorithms  and  name  lookup  techniques;  in  analyzing  their 
data  by  the  identification  of  weighting  factors  and  statistical 
sampling  for  determining  error  and  omission  rates;  and  lastly, 
predicting  the  accuracy  and  efficiency  of  candidate  retrieval  keys. 


Seymour  Jeffery,  Chief 
Systems  and  Software  Division 
Institute  for  Computer  Sciences 
and  Technology 
National  Bureau  of  Standards 


iii 


ACKNOWLEDGMENTS 


The  authors  would  like  to  express  their  appreciation  to  all  those 
who  contributed  to  the  preparation  of  this  document  through  their 
willingness  to  discuss  their  relevant  experience  and  expertise. 
Particular  thanks  are  due  two  members  of  the  NBS  Institute  for 
Computer  Sciences  and  Technology:    Dr.  Thomas  C.  Lowe,  who  coordin- 
ated this  effort  from  its  inception,  gave  incentive  and  direction 
to  the  project  while  assisting  in  every  way  possible  with  preparation 
of  the  final  report;  and  John  L.  Berg,  whose  comments  and  suggestions 
have  been  invaluable  throughout  the  course  of  the  project. 


iv 


TABLE  OF  CONTENTS 


Page 


1.0    Introduction   1 

1.1  The  Legislation   1 

1.2  Retrieval  of  Individual  Records   3 

1.3  Statistical  vs.  Dossier  Files   4 

1.4  Privacy  vs.  Security    6 

1.5  Summary   8 

2.0    State-of-the-Art  Retrieval  Techniques    9 

2.1  Introduction    9 

2.2  Varying  Selective  Values  of  Non-Unique  Identifiers  ...  10 

2.3  Importance  of  the  Name  Variable  in  Retrieval 

Through  Use  of  Non-Unique  Identifiers   12 

2.4  Selected  State-of-the-Art  for  Name  Lookup  Techniques  .  .  12 

2.5  Implementation  of  Name  Lookup  Techniques    26 

2.6  Other  Search  Strategies   29 

3.0    Identification  of  Weighting  Factors    32 

3.1  Accuracy  Constraints  Imposed  by  the  Privacy  Act   32 

3.2  Determination  of  Weighting  Factors  for  Existing  Files.  .  35 

3.2.1  Data  Base  Definition  and  Forms  Design  ......  36 

3.2.2  Data  Collection  Techniques   37 

3.2.3  Keying  of  Data  and  Input  Controls   42 

3.2.4  Computer  Edit  Routines   44 

3.2.5  Kiariual  Checks  for  Accuracy   ^7 

3.3  Summary  of  Weighting  Factors    49 

4.0    File  Validation   52 

4.1  Introduction   52 

4.2  Validation  Techniques   52 

4.3  File  Validation  Example  ,   53 

5.0    Computer  Simulation  of  Data  Base  Retrieval                            .  59 

5.1  Precision  and  Recall   59 

5.2  Query  Formulation  for  Best  Results   61 

5.3  Soft-Match  Techniques   62 

5.4  Example  of  Use  of  File  Validation  Summary  Data    62 

5.5  Precision  and  Probable  Match   66 

6.0    Conclusions  and  Recommendations    69 

6.1  Conclusions   69 

6.2  Recommendations   70 

6.2.1  Evaluation  of  Current  File  Access  Capability.  .  .  71 

6.2.2  Evaluation  of  the  Data  Base   71 

Appendix  I  -  Precision  Tables   73 


i 


V 


TABLE  OF  ILLUSTRATIONS 


Page 


Table  2.4-1    IBM  Alpha  Inquiry  System  Personal 

Name  Encoding  Algorithm  15 

Table  2.4-2    Western  Airlines  Surname  Match  Rating  Algorithm.  ...  18 

Table  2.4-3    Rules  for  Developing  Phonetic  Frequency  Codes  19 

Table  2.4-4    Letter  Distributions  for  Phonetic  Frequency 

Code  Name  Lookup  Technique  21 

Table  2.4-5    Illustration  of  the  Application  of  the  NYSIIS 

Coding  Techniques   22 

Table  2.4-6    NYSSIS  Name  Coding  Rules    23 

Form  3.1-1     Decay  Factors  of  Standard  Biographical 

Data  Elements  34 

Form  3.2.1-1  Inherent  Characteristics  of  Biographical 

Data  Elements  38 

Form  3.2.1-2  Data  Base  Definition  Weighting  Factors  for 

Biographical  Data  Elements  .....    39 

Form  3.2.2-1  Data  Collection  Procedure  Weighting  Factors  for 

Biographical  Data  Elements    43 

Form  3.2.3-1  Keying  and  Input  Control  Weighting  Factors 

for  Batch  Input  45 

Form  3,2.3-2  Keying  and  Input  Control  Weighting  Factors 

for  Interactive  Input  46 

Form  3.2.4-1  Computer  Editing  Weighting  Factors    48 

Form  3.2.5-1  Weighting  Factors  for  Manual  Accuracy  Checks    50 

Form  3.3-1     Cumulative  Weighting  Factors  for  Standard 

Biographical  Data  Elements    51 

Table  4.2-1    Suggested  Format  for  File  Validation  Record  54 

Table  4.3-1    Source  Documents  for  Ten  Sample  Records  from 

the  Dummy  Data  Base.  55 

Table  4.3-2    Computer  Listings  for  Ten  Sample  Records  from 

the  Dummy  Data  Base  56 

Table  4.3-3    File  Validation  Record  for  a  Sample  Record 

from  the  Dummy  Data  Base  57 

Table  4.3-4    Summary  of  Errors  and  Omissions  Detected  in 

Analysis  of  10%  of  the  File.  .......  .    58 

Table  5.3-1    Retrieval  Ratios  (Estimates  of  Proportion  of  File 

Retrieved)  63 

Table  5.3-2    Statistical  Classification  of  Surname  Records  as 

Compiled  from  Social  Security  Administration  Accounts.  64 

vi 


TABLE  OF  CONTENTS 
(continued) 

Page 

Appendix  II    -  A  Probabilistic  Formulation  of  the 

Non-Unique  Access  Problem    174 

Appendix  III  -  Soft-Match  Techniques    181 

Appendix  IV    -  FORTRAN  Source  Code  for  the  Probability  Mode.  ...  186 


vi  i 


ACCESSING  INDIVIDUAL  RECORDS  FROM 
PERSONAL  DATA  FILES  USING  NON-UNIQUE  IDENTIFIERS 


G.  B.  Moore,  J.  L.  Kuhns,  J.  L.  Trefftzs,  C.  A.  Montgomery 
Operating  Systems,  Inc. 


ABSTRACT 


j    The  Privacy  Act  of  1974  places  restrictions  on  the  Federal,  state,  and 
'    local  agencies'  use  of  the  Social  Security  account  number  as  an  identi- 
fier.   For  some  agencies,  compliance  will  involve  changes  in  implementa- 
tion of  retrieval  algorithms.    This  report  describes  methodology  applic- 
able to  these  changes  in  the  more  general  context  of  the  problem  of 
retrieving  individual  records  from  files  using  non-unique  identifiers. 
State-of-the-art  retrieval  techniques  are  discussed,  a  method  for  assign- 
ing reliability  weights  to  various  personal  data  elements  is  presented, 
file  validation  techniques  for  the  error  and  omission  rates  of  data  items 
are  suggested,  and  a  retrieval  probability  model  --  designed  to  show 
likelihood  of  retrieval  of  a  subject's  record  given  a  variety  of  popula- 
tions, combinations  of  identifiers,  and  error/omission  rates  --  is  des- 
cribed.   A  methodology  is  developed  for  forming  confidence  factors  from 
the  established  error/omission  rates  for  combinations  of  non-unique 
identifiers  that  are  candidates  for  use  as  retrieval  keys.    Use  of  these 
confidence  factors  as  indices  into  the  precision  tables  produced  by  the 
probability  model  is  described. 

Key  Words:    Data  retrieval;  file  validation;  name  lookup;  non-unique 

identifiers;  personal  data  files;  Privacy  Act,  probability 
model ;  retri  eval . 


1.0  INTRODUCTION 

1.1    The  Legislation 

The  Privacy  Act  of  1974  opens  with  the  words: 
"The  Congress  finds  that- 

(1)  the  privacy  of  an  individual  is  directly  affected  by  the 
collection,  maintenance,  use,  and  dissemination  of  personal 
information  by  Federal  agencies; 

(2)  the  increasing  use  of  computers  and  sophisticated  informa- 
tion technology,  while  essential  to  the  efficient  operations 
of  the  Government,  has  greatly  magnified  the  harm  to  indi- 
vidual privacy  that  can  occur  from  any  collection,  main- 
tenance, use,  or  dissemination  of  personal  information; 

(3)  the  opportunities  for  an  individual  to  secure  employment, 
insurance,  and  credit,  for  his  right  to  due  process,  and 
other  legal  protections  are  endangered  by  the  misuse  of 
certain  information  systems; 
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(4)  the  right  to  privacy  is  a  personal  and  fundamental  right 
protected  by  the  Constitution  of  the  United  States;  and 

(5)  in  order  to  protect  the  privacy  of  individuals  identified 
"in  information  systems  maintained  by  Federal  agencies,  it 

is  necessary  and  proper  for  the  Congress  to  regulate  the 
collection,  maintenance,  use,  and  dissemination  of  infor- 
mation by  such  agencies. "1 

There  has  been  increasing  public  concern  over  the  trend  toward  integra- 
tion and  centralization  of  data  banks  containing  personal  information. 
In  the  privacy  legislation.  Congress  has  explicitly  defined  the  rights 
of  citizens  whose  records  are  maintained  by  Federal  agencies  as  well  as 
the  responsibilities  of  the  agencies  handling  the  data.    In  addition, 
the  legislation  also  established  the  Privacy  Protection  Study  Commission 
which  is  specifically  authorized  to: 

"...make  a  study  of  the  data  banks,  automated  data  processing 
programs,  and  information  systems  of  governmental,  regional, 
and  private  organizations,  in  order  to  determine  the  standards 
and  procedures  in  force  for  the  protection  of  personal  infor- 
mation...   In  the  course  of  conducting  the  study... the  Commis- 
sion may  research,  examine,  and  analyze. . .the  use  of  social 
security  numbers,  license  plate  numbers,  universal  identifiers, 
and  other  symbols  to  identify  individuals  in  data  banks  and  to 
gain  access  to,  integrate,  or  centralize  information  systems 
and  files. "2 

Specific  prohibitions  regarding  use  of  universal  identifiers  will,  for 
the  most  part,  await  the  results  of  the  Privacy  Protection  Study 
Commission  study.    The  current  emphasis  is  rather  on  restricting  the 
data  maintained  to  that  directly  relevant  to  the  agency.  However, 
since  much  of  the  concern  over  use  of  universal  identifiers  has  been 
generated  by  the  use  of  the  Social  Security  account  numbers  and  their 
potential  for  linking  files,  the  Act  has  specifically  provided  against 
its  use  as  a  required  item  of  information  unless  a  Federal  statute 
specifying  such  a  disclosure  is  applicable. 

"It  shall  be  unlawful  for  any  Federal,  State  or  local  govern- 
ment agency  to  deny  to  any  individual  any  right,  benefit,  or 
privilege  provided  by  law  because  of  such  individual's  refus- 
al to  disclose  his  social  security  account  number. "3 


Privacy  Act  of  1974,  Public  Law  no.  93-579,  Section  1. 

Loc.  cit. 

Ibid. ,  Section  7. 
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Thus  the  effect  of  the  legislation  is  that  individuals  whose  records  are 
maintained  in  the  files  of  Federal  agencies  must  be  allowed  to  review 
their  own  records  on  request,  the  retrieval  must  be  possible  without  use 
of  the  Social  Security  account  number,  the  data  maintained  in  the  files 
must  be  timely,  and  only  data  necessary  in  the  performance  of  the  agency's 
mission  should  be  stored. 

For  agencies  that  have  relied  heavily  on  the  Social  Security  number  for 
purposes  of  retrieval,  this  requirement  will  necessitate  some  major  re- 
adjustments.   Further  restrictions  are  imposed  on  the  type  of  information 
that  can  be  solicited  from  individuals  who  have  requested  copies  of  their 
record  by  the  July  1,  1975  guidelines  from  the  Office  of  Management  and 
Budget,  "Guidelines  for  Implementing  Section  3  of  the  Privacy  Act  of 
1974,"  which  states  that  the  items  published  in  the  Federal  Register  by 
each  agency  must  include: 

"...the  information  necessary  to  identify  the  record.  Where 
the  system  employs  a  specialized  identification  scheme,  the 
individual  should  not  be  required  to  provide  such  a  number  or 
symbol  as  an  absolute  requirement,  although  the  individual 
might  be  requested  to  supply  it  if  he  or  she  can  reasonably 
be  expected  to  know  it.    Instead,  alternative  combinations 
of  personal  characteristics  may  be  used  to  identify  individ- 
uals who  may  have  lost,  forgotten,  or  are  unaware  of  their 
identification  numbers  or  symbols.    For  example,  the  combina- 
tion of  name,  date  of  birth,  or  place  of  birth,  and  fatiier's 
first  name  may  be  sufficient  to  identify  an  individual  with- 
out the  use  of  a  system  identification  number.""^ 

Thus,  although  use  of  record  identification  numbers,  driver's  license 
numbers,  etc.,  is  not  prohibited,  careful  attention  must  be  given  to  the 
selection  of  a  set  of  identifiers  that  will  be  considered  reasonable  in 
the  particular  environment. 

1.2    Retrieval  of  Individual  Records 

The  implications  of  these  restrictions  will  differ  from  agency  to  agency. 
It  is  important,  however,  that  those  charged  with  responsibility  for  the 
selection  and  implementation  of  identification  schemes  be  aware  of  the 
retrieval  effectiveness  of  non-unique  personal  identifiers.    The  potential 
for  retrieval  by  means  of  exclusive  use  of  combinations  of  these  keys 
should  be  understood,  and  the  relevance  of  this  potential  for  use  with 
currently  active  data  bases  should  be  examined.    Consideration  must  also 
be  given  to  the  question  of  the  reliability  of  the  data  contained  in 
these  files  since  the  success  of  the  retrieval  algorithm  depends  upon 
the  accuracy  of  the  data  base  itself.    Errors  may  have  been  introduced 
into  the  data  base  (during  data  collection,  data  entry,  data  update,  etc.). 


Office  of  Management  and  Budget  Circular  A- 108  and  accompanying  "Guide- 
lines for  Implementing  Section  3  of  the  Privacy  Act  of  1974,"  Federal 
Register,  9  July  1975,  Vol.  40,  No.  32,  pp.  28948-28978. 


with  the  result  that  a  request  from  an  individual  for  a  copy  of  his  re- 
cord might  fail  due  to  the  presence  of  erroneous  data  in  the  record 
itself.    In  this  respect,  retrieval  algorithms  based  on  Social  Security 
account  number  or  other  standard  universal  identifiers  have  advantages 
(it  is  easier  to  alert  clerical  staff  to  the  necessity  of  checking  one 
or  two  numbers  for  accuracy  than  it  is  to  train  them  to  be  careful  with 
all  biographical  data),  adequate  retrieval  is  certainly  possible  using 
non-unique  identifiers  such  as  name,  birthdate,  etc.    In  fact,  such 
retrieval  is  sometimes  possible  even  when  the  variables  most  closely 
identified  with  an  individual  are  removed  from  the  records.  Statisticians 
involved  in  the  collection  and  analysis  of  statistical  data  for  research 
purposes  have  been  concerned  with  the  problem  of  protection  of  the  in- 
dividual whose  data  is  maintained  in  the  files.    The  need  to  collect 
such  data  in  order  to  study  causes  of  disease,  poverty,  violence,  migra- 
tion and  other  sociological  phenomena  is  well  understood,  but  the  individ- 
uals involved  have  the  right  to  expect  that  information  given  in  con- 
fidence will  be  treated  with  integrity.    Dr.  Fellegi  of  the  Dominion 
Bureau  of  Statistics  in  Canada  has  summarized  the  problem: 

"The  public  benefits  indirectly  from  the  legitimate  uses  of 
statistics  by  governments,  businesses,  non-profit  organiza- 
tions, academic  users,  etc.;  yet,  the  public  is  also  concerned 
about  the  increasing  burden  of  providing  the  required  statis- 
tics and  about  the  real  or  imagined  possibility  of  the  misuse 
of  the  data  provided  by  them.    The  explosive  increase  in  the 
demands  for  more  statistics  can  only  be  met,  without  impossible 
response  burdens  being  put  on  the  public,  through  a  more  effect- 
ive exploitation  of  the  data.    This  increases  geometrically  the  ^ 
magnitude  of  the  problem  of  checking  tabulations  for  disclosure." 

The  seemingly  contradictory  requirements  of  protecting  individual  privacy 
while  at  the  same  time  providing  the  citizen  the  benefits  attendant  on  the 
introduction  of  automated  record  keeping  systems  create  a  burden  for  those 
responsible  for  maintaining  the  systems.    They  must  restrict  the  data 
retained  in  the  file  to  that  clearly  relevant  to  the  application,  but 
at  the  same  time  be  able  to  identify  correctly  all  individuals  concerned 
on  demand.    Data  must  be  available  for  research  efforts,  but  information 
maintained  to  support  these  efforts  must  be  restricted  and  accessing  of 
individual  records  carefully  monitored.    The  dilemma  thus  created  is 
discussed  further  in  the  next  section. 

1.3    Statistical  vs.  Dossier  Files 

Until  recently,  it  was  generally  believed  that  a  basic  distinction  ex- 
isted between  statistical  files  (those  with  names,  addresses,  etc., 
removed  from  the  individual  record)  and  dossier  files  (those  containing 
explicitly  descriptive  information  about  the  individuals).  Dossiers 


Fellegi,  I.  P.,  "On  the  Question  of  Statistical  Confidentiality," 
Journal  of  the  American  Statistical  Association,  67  (March  1972), 
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were  considered  dangerous  from  the  point  of  view  of  exposure  whereas 
statistical  files  were  not.    James  Martin,  in  discussing  this  distinc- 
tion, says: 

"It  is  important  to  separate  the  concept  of  statistical  files 
from  the  concept  of  dossiers  about  individuals.    From  the 
point  of  view  of  privacy  protection,  they  are  entirely  differ- 
ent.   Many  of  the  uses  to  which  the  information  in  government 
and  research  data  banks  will  be  put  are  statistical.  In 
statistical  files  the  identification  of  the  individual  should 
be  stripped  off.    Once  that  is  done  the  files  can  be  used  more 
freely.    It  is  still  necessary  to  make  sure  that  people  cannot 
be  identified  by  some  other  means,  such  as  home  location,  a 
high  salary,  the  fact  that  he  has  13  children  or  cats,  weighs 
300  pounds,  or  has  a  combination  of  factors  that  makes  him 
unique.    It  is  sometimes  possible  to  extract  information 
about  individual s  from  a  statistical  data  bank  by  asking 
combinations  of  questions.    Sometimes  a  large  number  of 
questions  would  be  needed  and  controls  can  be  devised  to 
prevent  such  interrogation.    The  controls  are  more  complex 
and  subtle  than  on  the  type  of  data  bank  in  which  details 
about  an  individual  are  requested  directly.    In  general, 
statistical  data  banks  will  not  reveal  the  identities  of 
the  persons  about  whom  data  are  recorded."^ 

Recent  research  indicates  that  the  data  contained  in  statistical  files  i 
not  necessarily  protected  from  unauthorized  disclosure.  Dr.  Schltirer  of 
the  Department  of  Medical  Statistics  at  the  University  of  Ulm  in  Germany 
gives  the  following  account  of  his  investigation  of  retrieval  from  inter 
active  data  banks. 

"Until  fairly  recently,  removing  name,  address,  and  --  per- 
haps --  date  of  birth  from  a  record  would  secure  anonymity 
in  most  cases.    The  advent  of  the  computer  has  changed  this 
situation.    It  was  soon  realized  that  the  distinction  between 
a  dossier  data  bank  (returning  full  identifying  information 
to  the  user)  and  a  statistical  data  bank  (providing  jnly 
numbers  of  persons)  is  largely  delusive,  if  the  user  is  allow- 
ed to  communicate  with  the  data  bank  by  dialogue. 

A  prerequisite  for  this  intrusion  by  dialogue  is  preknow- 
ledge.    The  intruder  must  have  some  a^  priori  information 
from  the  statistical  data  bank.    The  same  information  must 
be  stored  in  Mr.  X's  record,  so  that  preknowledge  and  the 
corresponding  part  of  the  record  can  be  matched.    The  pre- 
knowledge must  suffice  to  identify  or,  at  least,  nearly 
identify  Mr.  X.    An  identification  experiment  with  authentic 


Martin,  James,  Secu'^ity,  Accuracy,  and  Privacy  in  Computer  Systems, 
Prentice-Hall,  New  Jersey,  1973,  p.  432. 
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data  will  be  described  in  this  paper.    It  will  be  seen  that 
relatively  little  preknowledge  will  do  for  identification 
purposes  --  at  least  for  the  life  'attacked'  in  this  experi- 
ment. "7 

Not  only  does  Dr.  Schltirer's  work  make  it  clear  that  records  can  be  re- 
trieved through  the  use  of  non-unique  identifiers,  it  also  gives  added  ■ 
credence  to  the  requirement  that  only  relevant  data  be  maintained  in  the 
files,  since  inclusion  of  extraneous  data  makes  the  record  more  vulnerable. 

Further  indication  of  the  need  for  such  restrictions  comes  from  the  re- 
search in  record  linkage  carried  out  by  Drs.  Fellegi  and  Sunter.  They 
have  developed 

"...a  mathematical  model... to  provide  a  theoretical  framework 
for  computer-oriented  solution  to  the  problem  of  recognizing 
those  records  in  two  files  which  represent  identical  persons, 
objects  or  events  (said  to  be  matched)  ...A  comparison  is" to  j 
be  made  between  the  recorded  characteristics  and  values  in  j 
two  records  (one  from  each  file)  and  a  decision  made  as  to 
whether  or  not  the  members  of  the  comparison-pair  represent 
the  same  person  or  event,  or  whether  there  is  insufficient 
evidence  to  justify  either  of  these  decisions  at  stipulated 
levels  of  error... A  theorem  describing  the  construction  and 
properties  of  the  optimal  linkage  rule  and  two  corollaries 
to  the  theorem  which  make  it  a  practical  working  tool  are 
given. "8 

The  existence  of  such  techniques  for  linking  individual  records  on  the 
basis  of  non-unique  identifiers  again  po-ints  up  the  feasibility  of  imple- 
menting retrieval  algorithms  based  on  those  variables.    An  awareness  of 
the  availability  of  such  techniques  further  clarifies  the  nature  of  the  ! 
threats  that  currently  exist  to  individual  privacy.    It  also  emphasizes 
the  importance  of  the  provisions  for  relevancy  of  data  in  the  Privacy 
Act  since  obviously  the  more  data  available,  the  easier  such  linkages 
become.    Similarly,  the  necessity  for  building  audit  checks  into  the 
system  to  provide  a  means  for  determining  what  information  was  accessed 
by  whom  becomes  clear  in  the  light  of  currently  available  methodologies. 

1.4    Privacy  vs.  Security  j 

Privacy  has  been  perceptively  defined  by  Westin  as  the  right  "...to  g 
determine  what  information  about  ourselves  we  will  share  with  others," 


Schl5rer,  J.,  "Identification  and  Retrieval  of  Personal  Records  from  a 
Statistical  Data  Bank,"  Methods  of  Information  In  Medicine,  14  (1,  1975). 

'Fellegi,  I.  P.,  and  A.  B.  Sunter,  "A  Theory  for  Record  Linkage,"  Journal 
of  the  American  Statistical  Association,  (Dec.  1969),  pp.  1183-1210. 

Westin,  Alan,  as  quoted  by  I.  P.  Fellegi,  "On  the  Question  of  Statistical 
Confidentiality,"  op. ci t. ,  p.  18. 

6 


where  the  'others'  are  persons  or  agencies  who  are  presumably  authorized 
to  hold  and  use  such  information.    Thus  the  legislative  concept  of  pri- 
vacy in  effect  concerns  the  protection  of  information  about  individuals 
from  misuse  by  duly  constituted  authorities,  or  persons  or  agencies  to 
which  such  authority  has  been  delegated  (e.g.,  for  statistical  studies). 

Security,  on  the  other  hand,  is  concerned  with  protection  of  this  infor- 
mation from  unauthorized  'others.'    Such  unauthorized  access  may  be  de- 
liberate, or  inadvertent:  e.g.,  a  person  seeking  information  from  his  own 
record  may  be  given  the  dossier  of  another  individual  by  mistake. 

Protection  of  information  in  a  computerized  file  from  unauthorized  access 
is  partly  a  technical,  system-internal,  matter,  and  partly  a  procedural, 
system-external  matter.    System- internal  protections  include  obvious  items 
such  as  use  of  passwords  and  other  identifying  techniques  to  guard  against 
deliberate  access  attempts  by  unauthorized  persons.    More  fundamentally, 
such  protection  requires  a  high  degree  of  precision  in  the  retrieval  al- 
gorithm to  prevent  inadvertent  delivery  of  the  wrong  information  record 
to  a  particular  requestor  (as  well  as  erroneous  updates  and  purges). 

Procedural  protections  involve  basic  items  such  as  locks  and  security 
guards,  as  well  as  requests  for  proof  of  identity  from  persons  seeking 
information  about  their  own  records,  further  verification  based  on  ques- 
tions about  the  content  of  the  record  before  it  is  delivered  to  the  re- 
questing individuals,  and  so  forth.    In  the  case  of  a  computerized  infor- 
mation bank  where  requesting  individuals  are  allowed  access  via  a  computer 
terminal,  a  combination  of  technical  and  procedural  safeguards  is  required 
to  insure  protection  of  the  component  information  records. 

In  a  recent  article  on  computer  privacy  and  computer  security,  Willis 
Ware  states: 

"In  the  context  of  computer-based  systems,  the  matter  of  access 
control  is  an  essential  part  of  a  larger  issue  referred  to  as 
computer  security  which  can  be  defined  as: 

The  protection  of  the  equipment,  facilities  and  data 
of  an  information  system  against  deliberate  or  acci- 
dental damage,  and  against  denial  of  use  by  legitimate 
users,  together  with  the  assurance  that  information 
will  be  delivered  by  the  system  only  to  individuals 
authorized  to  receive  it."-'-'^ 

The  phrase  "legitimate  users"  of  an  information  system  implies  in  the 
privacy  context  that  legitimate  uses  of  personal  information  can  be  de- 
fined, and  that  "individuals  authorized"  to  access  such  information  will 
respect  the  right  to  privacy  of  those  persons  who  provided  it.    This  is 


Ware,  Willis  H. ,  "Computer  Privacy  and  Computer  Security,"  Bulletin  of 
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the  core  of  the  privacy  issue,  which  essentially  involves  the  loss  of 
control  over  personal  information  incurred  by  an  individual  in  providing 
such  data  to  some  agency  or  data  bank,  and  the  profound  effect  that  re- 
linquishing control  of  this  information  can  have  on  the  life  of  the 
particular  individual.    It  is  the  concern  of  those  involved  in  the  effort 
to  protect  the  privacy  of  the  individual  to  identify  measures,  legal, 
procedural,  and  technological,  by  which  potentially  detrimental  effects 
of  this  loss  of  control  can  be  nullified. 

1.5  Summary 

In  Section  1  some  of  the  difficulties  faced  by  those  responsible  for  the 
management  of  personal  data  bases  in  Federal  agencies  are  outlined.  Pro- 
blems inherent  in  the  provision  of  continued  service  with  no  attendant 
reduction  in  efficiency  due  to  the  increased  emphasis  on  the  protection 
of  individual  privacy  have  been  stressed.    The  remainder  of  this  report 
is  concerned  with  the  presentation  of  methodologies  potentially  useful 
in  the  minimization  of  these  difficulties. 

The  application  of  current  retrieval  technologies  to  the  problem  of 
retrieval  through  use  of  non-unique  personal  identifiers  is  the  basic 
methodology  under  consideration.    A  discussion  of  relevant  retrieval 
algorithms  is  given  in  Section  2.    Section  3  suggests  an  approach  to 
determination  of  the  accuracy  and  reliability  of  the  various  identifiers 
contained  in  current  files.    Forms  that  might  be  used  in  carrying  out  an 
analysis  procedure  are  introduced,  and  a  mechanism  for  arriving  at 
weighting  scores  for  the  individual  identifiers  is  described.    Section  4 
demonstrates  a  similar  mechanism  for  analyzing  error  and  omission  rates 
for  each  of  the  personal  identifiers  contained  in  the  file. 

Section  5  describes  the  probability  model  developed  to  assist  in  deter- 
mination of  the  best  combination  of  personal  identifiers  available  for 
use  as  retrieval  keys  given  the  idiosyncratic  error  and  omission  rates 
of  those  variables  within  a  particular  file.    An  example  of  the  use  of 
the  output  from  the  probability  model  in  making  this  determination  is 
given.    Conclusions  are  formulated  in  Section  6  which  also  contains  a 
summary  of  procedures  recommended  for  those  involved  in  the  analysis  of 
personal  data  files. 


Four  Appendices  are  included: 

Appendix  I:       Precision  Tables 

Appendix  II:     A  Probabilistic  Formulation  of  the  Non-Unique  Access 
Problem 

Appendix  III:    Soft  Match  Techniques 

Appendix  IV:     FORTRAN  Source  Code  for  the  Probability  Model. 
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2.0    STATE-OF-THE-ART  RETRIEVAL  TECHNIQUES 


2.1  INTRODUCTION 

The  personal  data  files  affected  by  the  provisions  of  the  Privacy  Act 
represent  a  wide  spectrum  of  applications  with  very  different  require- 
ments.   By  definition  the  files  all  contain  dossier-type  data,  but  they 
vary  in  terms  of  the  non-biographical  variables  maintained,  the  ulti- 
mate use  of  the  data,  the  frequency  of  file  access,  the  retrieval 
mechanisms  employed,  the  degree  to  which  accuracy  has  been  stressed, 
etc.    These  distinctions  reflect  the  differing  operational  requirements 
of  the  agencies  involved.    Clearly,  the  provisions  of  the  Privacy  Act 
giving  the  individual  access  to  his  own  records  will  cause  re-evaluation 
of  file  management  requirements  and,  in  many  cases,  some  system  changes 
will  be  necessary  in  order  to  make  this  data  available. 

A  clear-cut  requirement  for  some  system  changes  exists  when  the  retrieval 
keys  do  not  include  any  of  those  combinations  of  identifiers  that  can  be 
readily  recalled  by  the  individual.    Similarly,  systems  that  have  placed 
a  high  degree  of  reliance  on  the  use  of  the  Social  Security  account 
number  in  the  retrieval  algorithms  will  require  extension  of  these  system 
modifications  to  insure  effective  retrieval.    But  even  if  these  changes 
result  in  the  availability  of  a  standard  set  of  biographical  variables 
as  potential  retrieval  keys,  the  level  of  accuracy  maintained  for  those 
variables  may  fall  short  of  that  required  of  retrieval  keys.    It  is 
entirely  possible  that  a  higher  degree  of  accuracy  exists  for  the  vari- 
ables that  have  historically  been  used  as  retrieval  keys  than  for  those 
now  promoted  as  candidates  for  that  task.    Such  a  disparity  in  levels  of 
accuracy  could  have  resulted  from  awareness  on  the  part  of  the  clerical 
staff  of  the  need  to  verify  the  retrieval  keys,  or  from  the  programming 
of  special  checks  on  those  fields,  or  the  like.    Further,  this  skewing 
of  levels  of  accuracy  may  well  have  gone  unnoticed  since  the  retrieval 
algorithm  has  operated  sufficiently  well  to  fulfill  the  operational 
requirements  of  the  agency. 

To  illustrate  this  point,  consider  the  case  where  an  agency  uses  a 
specific  data  item,  for  example  AGE,  to  trigger  notification  of  the 
individual  concerning  a  change  in  status.    In  implementing  and  maintain- 
ing such  a  system,  the  emphasis  on  manual  and  system  checks  of  the  vari- 
ables BIRTHDATE  and  AGE  would  be  greater  than  that  on  NAME  and  ADDRESS. 
Indeed,  since  postmen  have  become  expert  at  detecting  near  matches  on 
name  and  street  numbers,  the  degree  of  accuracy  required  to  complete 
the  correspondence  is  indeed  less  than  that  required  for  retrieval  of 
the  record  on  the  basis  of  current  age. 

One  of  the  authors,  Mr.  Tref f tzs ,  has  experienced  this  phenomenon.  His 
is  an  unusual  name  that  causes  a  great  deal  of  confusion.    As  a  result 
it  is  seldom  spelled  correctly.    Mr.  Trefftzs  has,  in  recent  weeks, 
received  communications  from  a  variety  of  sources  (governmental,  maga- 
zines, professional  societies,  etc.)  in  which  his  name  was  consistently 
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misspelled  in  a  number  of  inconsistent  ways: 


TREVITZ     TRAFFT     TRAFFTZS     TRRFFTZA     TREFTZ  etc. 

These  misspellings  prevented  neither  the  retrieval  of  his  record  nor 
the  delivery  of  the  message.    Yet,  if  the  systems  involved  were  capable 
of  handling  retrieval  on  the  basis  of  biographical  data  including  name, 
and  he  were  to  request  a  copy  of  his  record    from  each,  it  is  likely 
that  some  of  the  systems  would  fail  to  locate  the  record,  given  the 
nature  of  the  error  that  exists  in  the  name  itself.    This  emphasizes 
the  importance  of  the  change  of  system  perspective  attendant  on  intro- 
duction of  new  accessing  requirements.    Problems  such  as  gross  mis- 
spellings of  names  may  block  retrieval  of  certain  records  until  the 
level  of  accuracy  of  the  NAME  variable  has  been  brought  within  reason- 
able bounds. 

Existing  file  organizations  and  retrieval  mechanisms  may  thus  be  entirely 
responsive  to  the  needs  of  a  particular  agency  while  being  inadequate  in 
terms  of  handling  the  new  requirements.    The  most  difficult  aspect  of 
this  problem  arises  from  the  fact  that  the  required  retrieval  capabili- 
ties may  not  have  been  built  into  the  original  system.    In  some  cases, 
extension  of  current  system  capabilities  will  be  sufficient  to  handle 
the  new  requirements;  in  others,  new  sets  of  capabilities  will  be  needed; 
still  other  situations  will  arise  where  restructuring  of  the  files  is 
required  in  order  to  make  this  data  available  in  the  required  way. 

Decisions  regarding  the  system  changes  required  will  be  based  on  several 
factors:    demand  (i.e.,  the  number  of  requests  for  this  information), 
capabilities  of  the  current  system  (some  will  be  set  up  in  such  a  way 
that  required  changes  will  be  minimal),  characteristics  of  the  data  base 
(number  of  non-unique  identifiers,  available  accuracy  rates,  typical 
problems  with  a  given  variable),  etc.    If  it  is  evident  that  the  complete 
redesign  of  the  system  is  required  in  order  to  meet  the  new  regulations, 
the  agency  will  want  to  examine  data  base  structures  and  retrieval 
techniques  that  might  assist  in  optimizing  the  new  system.    The  re- 
mainder of  this  section  is  devoted  to  a  description  of  the  current 
state-of-the-art  for  retrieval  techniques  as  they  apply  to  the  problem 
of  accessing  data  using  non-unique  identifiers.    Since  there  is  no 
single  answer  to  the  implementation  of  these  requirements,  this  section 
is  presented  in  the  form  of  a  survey  in  the  hope  that  the  availability 
of  this  information  will  clarify  the  options  available  to  those  con- 
fronted with  the  problem. 


2.2   VARYING  SELECTIVE  VALUES  OF  NON-UNIQUE  IDENTIFIERS 

Some  of  the  identifiers  available  in  personal  data  files  are  highly 
individualized.    The  variable  NAME  is  the  most  obvious  example  of  this. 
Other  data  items  for  example,  SEX,  are  binary,  and  therefore  not  particu- 
larly useful  in  identifying  an  individual  record.    Yet  collection  of 
even  the  most  straightforward  items  can  be  difficult  since  the  items 
requested  are  sometimes  subject  to  vastly  different  interpretations. 

10 


There  is  a  natural  tendency  to  assume  that  basic  biographical  data  is 
easily  obtainable,  and  that  standard  definitions  can  be  taken  for  grant- 
ed.   However,  in  actual  data  collection  situations,  surprising  misunder- 
standings can  and  do  occur  and,  most  importantly,  are  often  not  detected 
until  much  later.    The  result  is  that  variables  considered  to  be  reliable 
in  the  file  are  many  times,  in  reality,  only  marginally  so. 

An  interesting  discussion  of  the  problems  inherent  in  obtaining  biograph- 
ical data  is  given  in  the  Uniform  Hospital  Discharge  Data  Demonstration 
Summary  Report: 

"DATE  OF  BIRTH:  Date  of  birth  is  usually  an  easy  item 
to  collect  on  admission.  However,  there  are  instances 
of  elderly  persons  not  remembering  their  exact  date  of 
birth,  and  others  who  provide  incorrect  dates  for  per- 
sonal reasons. 

SEX:    The  incidence  of  encountering  a  patient  with  true 
indeterminate  sex  status  is  very  low.    Most  hospitals 
feel  the  indeterminate  category  is  unnecessary,  but 
hospitals  who  perform  trans-sexual  operations  find  it  a 
useful  item. 

MARITAL  STATUS:    While  this  item  is  usually  easy  to  ob- 
tain, there  are  some  inherent  difficulties  in  assuring 
validity,  and  these  include: 

Persons  living  in  common-law  relationships  who  do 
not  wish  to  be  classified  under  any  of  the  alterna- 
tives. 

Divorced  persons  who  consider  themselves  single 
and  state  so  at  admission. 

Single  females  admitted  for  childbirth  do  not 
generally  want  to  be  identified  as  single  and  state 
they  are  married. 

The  original  intent  of  this  item  was  to  attempt  to 
secure  some  indication  of  the  living  arrangement 
of  a  patient,  but  it  proves  to  be  of  marginal  utili- 
ty for  this  purpose.    This  is  a  complex  variable 
to  establish  if  the  many  socio-economic  factors 
which  enter  into  the  underlying  health  care  implica- 
tions associated  with  living  arrangements  are  con- 
sidered. "H 


Hodgson,  D.  A.,  L.  E.  Kucken,  and  J.  M.  Ensign,  The  Uniform  Hospital 
Discharge  Data  Demonstration,  Summary  Report,  Health  Service  Foundation, 
Chicago,  1973,  pp.  30-31. 
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2.3    IMPORTANCE  OF  THE  NAME  VARIABLE  IN  RETRIEVAL  THROUGH  USE  OF 
NON-UNIQUE  IDENTIFIERS 


Use,  as  well  as  inherent  variability,  of  identifiers  will  vary  greatly 
from  one  application  to  the  next.    The  variability  within  groups  of 
identifiers  is  probably  most  extreme  for  the  NAME  variable.    The  fol- 
lowing example  of  such  variability  comes  from  a  bibliographic  data  base 
application.    The  report  states  that: 

"...many  of  the  individual  data  elements  exhibit  great 
variety  (i.e.,  lists  of  their  contents  are  extensive), 
and  show  relatively  disparate  distributions.    This  be- 
havior is  encountered  in  different  degrees  in  regard 
to  items  such  as  words  in  the  titles  of  monograph  or 
periodical  articles,  assigned  subject  headings,  authors' 
names  and  citations.    ...In  general,  the  distributions 
are  approximately  hyperbolic,  so  that  a  small  portion 
of  items  may  account  for  a  substantial  proportion  of 
occurrences,  while  the  majority  of  items  occur  only 
infrequently.  -...Of  all  the  data  elements,  personal 
author  names  exhibit  a  distribution  which  is  at  its 
most  extreme  in  one  direction.    As  is  shown  ...the 
most  frequent  (full  name)  author  name  in  a  file  of 
50,000  names  occurred  only  sixteen  times,  while  over 
35,000  of  the  names,  or  over  70  percent  of  the  file, 
occurred  once  only."^^ 

The  same  tables  show  that,  with  respect  to  surnames  onlyj20,000  or  40% 
of  the  file,  occurred  only  once.    These  figures  emphasize  the  high 
selective  value  of  NAME  as  an  identifier  and  thus  provide  the  rationale 
for  the  following  consideration  of  name  lookup  techniques. 


2.4    SELECTED  STATE-OF-THE-ART  NAME  LOOKUP  TECHNIQUES 

A  majority  of  the  name  lookup  algorithms  currently  in  use  are  based  on 
a  phonetic  scheme  aimed  at  minimizing  the  common  transcription  problems 
in  the  recording  of  names.    The  IBM  Alpha  Search  Inquiry  Program  des- 
cription states  that: 

"Using  the  phonetic  approach  for  personal  name  increases 

the  user's  ability  to  access  the  alpha  search  record 

even  though  the  exact  spelling  of  a  name  may  not  be  known. 


D.  W.  Fokker  and  M.  F.  Lynch,  "Application  of  the  Variety-Generator 
Approach  to  Searches  of  Personal  Names  in  Bibliographic  Data  Bases- 
Part  I.    Microstructure  of  Personal  Authors'  Names,"  Journal  of 
Library  Automation,  7  (2  June  1974),  pp.  105-117. 
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Thus  the  effects  of  transcription  errors,  partially 
illegible  signatures  on  correspondence,  and  sound- 
alike  names  can  be  minimized, "-^^ 

In  analyzing  name  lookup  techniques,  it  is  useful  to  refer  to  the  in- 
formation systems  concepts  of  precision  and  recall.    Recall  represents 
the  number  of  relevant  items  contained  in  a  given  file  which  are  re- 
trieved in  response  to  a  particular  query,  while  precision  refers  to 
the  number  of  file  items  retrieved  which  are  actually  relevant  to  the 
query  or  search  prescription. 

In  general,  the  objective  of  system  improvement  is  to  increase  the  de- 
gree of  precision  or  reliability,  without  substantially  increasing  the 
amount  of  irrelevant  material  recalled  or  selected  from  the  file. 

In  a  document  retrieval  system,  however,  it  is  rare  (in  practice,  at 
least)  that  there  is  a  single  file  item  which  is  capable  of  satisfying 
a  given  query.    Generally  speaking,  several  documents  will  be  relevant 
to  a  query,  where  the  degree  of  relevance  is  defined  concretely  in  terms 
of  the  concepts  used  in  the  query  and  abstractly  in  terms  of  actual 
utility  to  the  requestor. 

Conversely,  in  a  name  retrieval  system,  the  usual  case  is  that  there  is 
a  unique  file  item  which  satisfies  a  query,  although  the  existence  of 
other  items  listed  under  alternate  spellings  of  the  name  must  also  be 
assumed  by  the  search  strategy.  This  is  a  critical  issue,  which  does 
not  appear  to  be  effectively  handled  by  many  of  the  name  search  tech- 
niques investigated  in  the  course  of  this  study. 

In  evaluating  name  lookup  techniques  for  which  complete  documentation  is 
available,  there  are  two  essential  criteria  for  determining  relative 
effectiveness.    Both  of  these  may  seem  obvious,  but  since  several  of  the 
lookup  techniques  described  here  fail   on  one  or  both  counts,  the  topic 
merits  discussion. 

The  first  basic  question  is  simply:    does  the  strategy  allocate  similar 
names  to  the  same  logical  group  --  and  more  particularly,  are  alternate 
versions  of  the  same  name  (Heinz,  Heintze)  in  the  same  logical  group? 
The  answer  is  often  in  the  negative,  and  this  is  the  primary  means  of 
effecting  an  improvement  in  reliability  of  a  given  method,  as  we  shall 
consider  below. 

The  second  basic  evaluation  criterion  is  the  converse  of  the  first: 
does  the  strategy  allocate  dissimilar  names  to  different  logical  groups? 
Although  the  possibility  of  violating  this  principle  appears  unlikely, 
many  name  lookup  techniques  do  in  fact  fail  to  satisfy  this  criterion. 


Alpha  Search  Inquiry  System  General  Infonnation  Manual,  GH20- 1188-3, 


IBM  Corporation,  White  Plains,  N.Y.,  April  1974,  p.  7. 
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including  Soundexj  the  first  widely  used  phonological ly  based  system  for 
name  encoding.    Most  of  the  systems  currently  in  use  have  been  influenc- 
ed by  the  rules  established  50  years  ago  by  Margaret  Odell  and  Robert 
Russell  for  the  Soundex  system. 

Knuth  summarizes  the  original  Soundex  rules  as  follows: 

"1.    Retain  the  first  letter  of  the  name,  and  drop  all 
occurrences  of  a,  e,  h,  i,  o,  u,  w,  y  in  other 
positions. 

2.    Assign  the  following  numbers  to  the  remaining  letters 


3.  If  two  or  more  letters  with  the  same  code  were 
adjacent  in  the  original  name  (before  Step  1), 
omit  all  but  the  first. 

4.  Convert  to  the  form  "letter,  digit,  digit,  digit" 
by  adding  trailing  zeros  (if  there  are  less  than 
three  digits),  or  by  dropping  rightmost  digits 
(if  there  are  more  than  three). "14 

This  algorithm  fails  to  meet  the  evaluation  criteria  described  above,  as 
Knuth  has  pointed  out: 

"...the  names  Euler,  Gauss,  Hilbert,  Knuth,  Lloyd,  and 
Lukasiewicz  have  the  respective  codes  E460,  G200,  H416, 
K530,  L300,  L222.    Of  course  this  system  will  bring  to- 
gether names  that  are  somewhat  different,  as  well  as 
names  that  are  similar;  the  same  six  codes  would  be  ob- 
tained for  Ellery,  Ghosh,  Heilbronn,  Kant,  Ladd,  and 
Lissajous.    And  on  the  other  hand  a  few  related  names 
like  Rogers  and  Rodgers,  or  Sinclair  and  St.  Clair,  or 
Tchebysheff  and  Chebyshev,  remain  separate.    But  by  and 
large  the  Soundex  code  greatly  increases  the  chance  of 
finding  a  name  in  one  of  its  disguises. "15 

The  type  of  refinement  that  has  taken  place  in  development  of  name  look- 
up techniques  is  obvious  from  comparing  the  results  of  applying  the  rules 
used  in  the  IBM  Alpha  Search  Inquiry  System  (Table  2.4-1)  with  those  of 
Soundex,  using  two  of  the  examples  mentioned  above. 


Knuth,  Donald  E.,  The  Art  of  Computer  Programming,  Vol.  Ill:  Sorting 
and  Searching  ,  Addison-Wesley,  Reading,  Mass.,  1973,  p.  392. 


after  the  first: 


b,  f ,  p,  V  ^  1 

c,  g,j,k,  q,  s,x,  z-^2 

d,  t  ^  3 


1  ^  4 
m,  n  ^  5 
r  6 


15 


Ibid. 
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Table  2.4-1    IBM  Alpha  Inquiry  System  Personal  Name  Encoding 
Algorithm  1^ 


This  routine  builds  a  phonetic  key  from  the  last  name  passed  by  the 
calling  program.    The  phonetic  key  is  7  packed  bytes  (14  digits).  The 
routine  uses  two  tables,  a  first  character  table  and  a  basic  table. 
The  first  character  table  (see  below)  is  used  to  recognize  first  letters 
or  letter  combinations  and  give  them  significance.    For  example,  A  in 
Armour  has  a  value  of  1  to  be  put  in  the  key.    A  in  Farmer  has  no  value 
and  is  ignored.    If  the  first  letter  or  letter  combination  does  not 
appear  in  the  first  character  table,  a  zero  is  inserted  in  the  first  key 
position  and  the  basic  table  is  used  to  code  the  first  character  into 
the  record  key  position. 

The  basic  table  (see  below)  is  used  to  recognize  letters  or  letter 
combinations  that  are  phonetically  equivalent.    Vowels  and  the  letters 
H,  W,  Y  are  ignored.    Letters  with  the  same  phonetic  value  which  are 
adjacent  to  each  other  are  treated  as  single  letters.    For  example,  in 
CK  and  TT  the  second  character  is  dropped.    However;  if  a  third  charac- 
ter has  the  same  value  as  the  previous  two,  it  is  retained.    The  re- 
mainder of  any  unfilled  key  is  filled  with  zeroes. 

The  first  character  table  used  in       The  basic  table  used  in  this 
this  routine  is:  routine  is: 


Letter(s) 

Value 

Code 

Letter(s) 

A 

1 

0 

Z,S,CI,CY,CE,TS,TZ 

E 

1 

1 

D,T 

GF 

08 

2 

N 

GM 

03 

3 

M 

GN 

02 

4 

R 

H 

2 

5 

L 

I 

1 

6 

J,SH,SCH,CH 

J 

3 

7 

C,G,K,Q,X,DG 

KN 

02 

8 

F,V,PH 

0 

1 

9 

B,P 

PF 

08 

PN 

02 

PS 

00 

U 

1 

W  (except  WR) 

4 

WR 

04 

Y 

5 

^^Ipha  Search  Inquiry  System  General  Information  Manual,  IBM  Corpora- 
tion, White  Plains,  New  York,  GH20-1188-3,  1974,  pp. 41-42. 
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Table  2.4-1    IBM  Alpha  Inquiry  System  Personal  Name  Encoding 
Algorithm  (Continued) 16 


EXCEPTIONS  TO  BASIC  TABLE  PROCESSING* 


In  certain  situations  where  letters  or  groups  of  letters  have  multiple 
sounds,  a  second  and  third  pass  through  name  encoding  is  made. 
Example:    CH  Has  a  hard  sound  in  Nichols,  a  soft  sound  in  Chavez. 


1st  Pass 


(Coded  in  Record) 

2nd  Pass 

3rd  Pass 

cz 

70 

6 

0 

CH 

6 

70 

0 

CK 

7 

7 

6 

C 

7 

7 

6 

K 

7 

7 

6 

DS 

0 

10 

10 

DZ 

0 

10 

10 

TS 

0 

10 

10 

TZ 

0 

10 

10 

Following  are  some  examples  of  the  results  of  coding  personal  names: 


NAME 

H 

A  R 

P 

E 

R 

KEY 

2 

4 

9 

4 

=  24940000000000 

NAME 

C 

0  L 

L 

I 

E  R 

KEY 

07 

5 

4 

=  07540000000000 

NAME 

S 

C  H 

U 

L 

T  Z 

KEY 

06 

5 

0 

=  06500000000000 

NAME 

L 

I  V 

I 

N 

G  S  T  0  N 

KEY 

05 

8 

2 

7  0  1  2 

=  05827012000000 

*Occurrences  of  the  letter  combinations  in  this  table  cause  generation 
of  alternate  phonetic  keys.    When  this  happens,  the  lookup  algorithm 
will  use  both  phonetic  keys  in  the  search  for  potential  matches.  This 
procedure  alleviates  many  of  the  problems  attributable  to  phonetic  miss- 
spellings  (e.g.,  Ohrbock  for  Ohrbach).    A  cross-reference  table  is  used 
to  connected  sound-alike  names  (e.g.,  Lyle  and  Lisle)  which  are  not 
handled  by  this  algorithm. 
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SEARCH  CRITERION 

SURNAME 

ASSIGNED  VALUES 

SOUNDEX 

ALPHA 

Assign  similar 
names  to  the 
same  logical  group 

RODGERS 

R326 

04740000000000 

\j  1  /  ~  \j  \j  \j  \j  \j  \j  \j  \j  \j  \j 

ROGERS 

R262 

04740000000000 

Assign  dissimilar 
names  to  differ- 
ent logical  group 

KANT 

K530 

02100000000000 

KNUTH 

K530 

07210000000000 

In  both  these  cases,  the  Alpha  Search  Inquiry  System  has  solved  the  pro- 
blem.   There  are,  however,  an  equally  good  set  of  examples  for  problems 
that  still  remain.    Thus,  although  this  algorithm  represents  progress 
over  that  used  in  the  Soundex  system,  it  is  far  from  perfect. 

Sometimes  quite  straightforward  algorithms  are  capable  of  handling  this 
problem  successfully  for  files  of  limited  extent.    The  match  rating 
approach  used  by  Western  Airlines  (see  summary  in  Table  2.4-2)  would 
correctly  identify  RODGERS  and  ROGERS  as  a  potential  match.    In  the 
Western  system,  the  matching  algorithm  would  compare  ROGERS  with  RGRS. 
Thenumber  of  unmatched  characters  would  be  1,  the  resulting  similarity 
rating  would  be  5,  and,  hence,  this  record  would  be  flagged  for  retrieval. 

On  the  other  hand,  this  algorithm  would  also  consider  KANT  to  be  a  like- 
ly match  for  KNUTH,  which  points  up  the  limitations  of  the  system.  It 
is  used  successfully  in  this  application  because  only  a  small  file  is 
searched  at  a  time.    Ms.  Gail  Hogan,  Manager  of  Passenger  Service 
Systems  and  Programming  for  Western,  states: 

"Since  our  reservations  system  is  flight/date  oriented 
(i.e.,  we  limit  the  scope  of  our  search  by  date  and/or 
flight  number),  we  have  little  difficulty  in  obtaining 
an  exact  name  match.  However,  if  there  are  passengers 
on  a  flight  with  the  same  surname,  we  display  a  list 
of  these  to  the  agent.  If  an  exact  match  is  not  made, 
we  continue  processing. .. in  order  to  compile  a  similar 
surname  list. "17 

Another  example  of  failure  to  meet  the  criterion  of  assigning  dissimilar 
names  to  different  buckets  can  be  given  using  the  Standardized  Phonetic 
Frequency  Code  algorithm  (see  Table  2.4-3)  as  described  in  the  NYSIIS 
evaluation  of  name  search  techniques. 1^    In  this  system,  an  attempt  was 


Personal  Communication  from  Gail  Hogan,  Manager  Passenger  Service 
Systems  and  Programming  for  Western  Airlines,  September  15,  1975. 

'Taft,  Robert  L.,  Name  Search  Technique,  New  York  State  Identification 
and  Intelligence  System,  August  1970. 
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Table  2.4-2    Western  Airlines  Surname  Match  Rating  Algorithm 


The  input  surname  and  each  of  the  PNI  [Personal  Numeric  Identifiers]  - 
item  surnames  are  encoded  by  this  program  prior  to  comparison.  Encoding 
is  as  follows: 

(1)  Deletion  of  all  vowels  unless  the  vowel  is  the  first  character  of 
the  surname. 

(2)  Elimination  of  all  double  consonants  by  deleting  the  second  contig- 
uous usage  of  any  consonant. 

(3)  Reducing  all  encoded  names  to  a  maximum  of  six  characters.  This 
is  done  by  retaining  the  first  three  and  last  three  encoded  char- 
acters. 

The  lengths  of  each  pair  of  encoded  names  (input  and  PNI  item)  are 
examined.*    If  they  differ  in  length  by  more  than  two,  no  similarity 
comparison  is  performed.    A  minimum  acceptable  similarity  rating  is 
established  for  each  pair  of  encoded  names  as  follows: 

Sum  of  lengths  is  4  or  less;  rating  of  5 

Sum  of  lengths  is  7  or  less;  rating  of  4 

Sum  of  lengths  is  11  or  less;  rating  of  3 

Sum  of  lengths  is  12;  rating  of  2 

The  minimum  acceptable  rating  establishes  which  PNI  surnames  are  not 
"similar  enough"  to  be  considered  by  the  agent. 

Comparison  of  encoded  names  is  then  performed.    This  comparison  is  from 
left  to  right,  character  by  character.    Matching  pairs  of  characters  are 
deleted.    This  comparison  continues  until  either  encoded  name  has  no  re- 
maining characters.    All  unmatched  characters  in  both  encoded  names  are 
packed  to  the  right  and  comparison  proceeds  from  right  to  left.  On 
completing  these  comparisons  the  number  of  unmatched  characters  in  the 
longer  name  is  subtracted  from  a  value  of  six  with  the  result  being  the 
similarity  rating  for  that  PNI  item. 

Each  PNI  item  that  has  a  similarity  rating  equal  to  or  greater  than  the 
minimum  rating  established  for  this  time  will  be  added  to  the  Retrieval 
Control  Record  [RCR].    This  program  will  add  to  the  RCR  until  the  record 
is  filled.    At  that  point  additional  entries  will  replace  other  items  in 
the  RCR  wnich  have  lower  similarity  ratings.    If  no  item  with  a  lower 
rating  is  found,  the  PNI  item  will  be  ignored. 

*That  is,  the  length  of  the  encoded  input  name  is  compared  with  that  of 
the  encoded  name  in  the  file. 


Gail  Hogan,  op. ci t. 
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Table  2.4-3    Rules  for  Developing  Phonetic  Frequency  Codes 


NAME  FIELD 

J.KUHNS 

G  ALTSCHULER 

CODE 

1,    In  the  name  field,  convert  DK  to  K, 
DT  tn  T    Sr  to  S    KN  to  N  and  MN  to  N 

J . KUHNS 

G  ALTSHULER 

2.    In  the  name  field,  replace  multiple 
letters  with  a  single  letter. 

J.KUHNS 
G.ALTSHULER 

?      Rpmnvp  vnwpl  <^    W    H    and  Y  but  keen 
the  first  letter  in  the  name  field. 

J .  KNS 
G.ALTSLR 

4.    The  first  digit  of  the  code  is  obtained 
using  PFl*and  the  first  letter  of  the 
name  field.    Remove  this  letter  after 
rndina 

J.NS 
G.LTSLR 

1 
3 

5.    Using  the  last  letters  of  the  name, 
use  Table  PF3*to  obtain  the  second 

UIUIL    Ui       LMC     L«L/VJC*                       0.0     IMQIljr  ICO 

t^rs  as  possible  and  remove  after 
coding. 

J.N 

G.LTSL 

16 
35 

6.    The  third  digit  is  found  using  Table 
PF2*and  the  first  character  of  the 

■Ft  vc  1"   n^mo        RpmnuP  fl'fl'PV^  mHinfl 

N 

LTSL 

157 
357 

7.    The  fourth  digit  is  found  using 
Table  PF2*and  the  first  character 
of  the  name  field.    If  no  letters 
remain  use  zero.    After  coding  remove 
the  letter. 

TSL 

1676 
3579 

o.     me  TTTtn  digit  is  touna  in  tne  same  _ 
manner  as  the  fourth  using  the  re- 
maining characters  of  the  name  field, 
if  any. 

SL 

16760 
35797 

*Defined  in  Table  2.4-4 
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made  to  achieve  a  uniform  distribution  of  codes  across  a  set  of  logical 
groups.    But  in  the  actual  implementation,  some  phonetically  dissimilar 
symbols  were  assigned  the  same  Phonetic  Frequency  code  value  as  shown 
in  Table  2.4-4. 

Applying  the  rules  shown  in  Table  2.4-3,  the  names  J.  Kuhns  and  6. 
Altshuler  are  reduced  to  the  codes  16760  and  35797,  respectively.  How- 
ever, as  shown  by  the  groups  of  letters  associated  with  the  same  codes 
in  Table  2.4-4,  'T.  Vines'  would  also  reduce  to  16760,  and  'J.  Butler' 
to  35797.    Moreover,  although  these  quite  dissimilar  names  would  be 
represented  by  the  same  code,  'Kuntz'  --  a  variant  of  'Kuhns'  --  would 
not. 

Returning  to  the  issue  of  coding  similar  names  such  that  they  are  al- 
located to  the  same  logical  group,  it  is  clear  that  this  principle  is 
especially  critical  for  alternate  spellings  of  the  same  name.  This 
logic  is  based  on  the  assumption  that  non-detectable*  errors  in  record- 
ing the  name  of  'Kuhns'  may  write  'Kuntz'  instead.    Also,  persons 
changing  their  name  to  a  more  angl icized  orthography  and  having  file 
records  listed  under  both  spellings  should  be  more  easily  traceable 
if  additional  phonological-to-orthographic  coding  rules  were  introduced. 

The  NYSIIS  coding  technique  introduces  a  few  such  rules  --  e.g.,  KN  ->  N, 
Z     S,  as  well  as  a  number  of  ad  hoc  rules  to  achieve  greater  reliabil- 
ity for  the  New  York  State  Criminal  justice  name  inventory.  However, 
many  of  these  rules  are  especially  aimed  at  a  large  population  of  Span- 
ish surnames  and  may  at  best  not  be  useful  for  files  with  small  popula- 
tions of  Spanish  surname  records.    At  worst,  they  may  introduce  un- 
desirable ambiguities,  as  in  the  case  of  'Risque'  in  Table  2.4-5,  which 
illustrates  the  application  of  the  NYSIIS  coding  rules  given  in  Table 
2.4-6  to  a  list  composed  mainly  of  OSI  employees. 

Again,  'Kuhns'  and  'Kuntz'  would  result  in  different  codes,  whereas  the 
dissimilar  Chinese  surnames  Li,  Lu,  Low,  Liu,  Lao  would  all  result  in 
the  same  code,  'L' . 

There  is  obviously  still  room  for  improvement,  via  the  introduction  of 
phonological-to-orthographic  rules  based  on  linguistic  --  rather  than 
ad  hoc  --  considerations.    Moreover,  such  rules  should  be  developed  with 
a  view  to  accommodating  a  variety  of  ethnic  surnames  used  in  the  United 
States  without  biasing  the  coding  techniques  such  that  rules  derived  to 
handle  one  type  of  surname  cause  ambiguous  coding  of  other  surname  types. 


*This  is  based  on  the  assumption  that  most  typographical  errors  can  be 
detected  by  comparing  new  names  to  the  existing  name  list  for  a  file 
and  generating  a  computer  listing  of  unique  occurrences  for  manual  in- 
spection.   This  of  course  assumes  an  investment  of  computer  and  human 
resources  for  error  detection  and  correction. 
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Table  2.4-4    Letter  Distributions  for  Phonetic  Frequency  Code  Name 
Lookup  Technique 
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Table  2.4-6    NYSIIS  Name  Coding  Rules 


1.  If  the  first  letters  of  the  name  are 
'MAC  then  change  these  letters  to  'MCC 
'KN'  then  change  these  letters  to  'NN" 
'K'  then  change  this  letter  to  'C 
'PH'  then  change  these  letters  to  'FF' 
'PF'  then  change  these  letters  to  "FF' 
'SCH'  then  change  these  letters  to  'SSS' 

2.  If  the  last  letters  of  the  name  are 
'EE'    then  change  these  letters  to  'Y^' 
'IE'    then  change  these  letters  to  'Y|5' 
'DT'  or  'RT'  or  'RD'  or  'NT'  or  'ND' 

then  change  these  letters  to  'D^' 

3.  The  first  character  of  the  NYSIIS  code  is  the  first  character  of 
the  name. 

4.  In  the  following  rules,  a  scan  is  performed  on  the  characters  of 
the  name.    This  is  described  in  terms  of  a  program  loop.    A  pointer 
is  used  to  point  to  the  current  position  under  consideration  in  the 
name.    Step  4  is  to  set  this  pointer  to  point  to  the  second  charac- 
ter of  the  name. 

5.  For  each  successive  position  of  the  pointer,  only  one  of  the  follow- 
ing statements  can  be  executed. 

a.  If  blank  go  to  rule  7. 

if  the  current  position  is  a  vowel  (AEIOU) 

and  if  it  is  E  followed  by  V  then  change  to  'AF' 

otherwise  change  current  position  to  'A'. 

b.  If  the  current  position  is  the  letter 
'Q'  then  change  the  letter  to  'G' 

'Z'  then  change  the  letter  to  'S' 
'M'  then  change  the  letter  to  'N' 

c.  If  the  current  position  is  the  letter  'K' 
and  if  the  next  letter  is  'N'  then  replace 
the  current  position  by  'N'  otherwise  replace 
the  current  position  by  'C 

d.  If  the  current  position  points  to  the  letter  string 
'SCH'  then  replace  the  string  with  'SSS' 

'PH'  then  replace  the  string  with  'FF' 

e.  If  the  current  position  is  the  letter  'H'  and  either 
the  preceding  or  following  letter  is  not  a  vowel  (AEIOU) 
then  replace  the  current  position  with  the  preceding  letter. 
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Table  2.4-6    NYSIIS  Name  Coding  Rules  (Continued) 


f.    If  the  current  position  is  the  letter  'W  and  the  preceding 
letter  is  a  vowel  then  replace  the  current  position  with  the 
preceding  position. 

If  none  of  these  rules  applies,  then  retain  the  current  posi- 
tion letter  value. 

6.  If  the  current  position  letter  is  equal  to  the  last  letter  placed 
in  the  code  then  set  the  pointer  to  point  to  the  next  letter  and  go 
to  Step  5. 

The  next  character  of  the  NYSIIS  code  is  the  current  position 
letter. 

Increment  the  pointer  to  point  at  the  next  letter. 
Go  to  Step  5. 

7.  If  the  last  character  of  the  NYSIIS  code  is  the  letter  'S'  then 
remove  it. 

8.  If  the  last  two  characters  of  the  NYSIIS  code  are  the  letters  'AY' 
then  replace  them  with  the  single  character  'Y'. 

9.  If  the  last  character  of  the  NYSIIS  code  is  the  letter  'A'  then 
remove  this  letter. 


The  NYSSIS  coding  rules  are,  in  several  instances,  subject  to  different 
interpretations.    As  a  result,  implementation  in  software  may  differ 
slightly  from  one  system  to  another.    This  illustration  demonstrates  the 
results  of  one  such  implementation. 
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An  example  of  a  pragmatically  developed  name  lookup  technique  is  that  in 
use  at  the  Medical  Information  Bureau.    In  this  system,  neither  a 
Soundex  nor  a  phonetic  technique  is  used.    Robert  W.  Keighley,  Director 
of  Systems  and  Planning  at  the  Recording  and  Statistical  Division  of 
Sperry  Rand,  the  servicing  agent  for  the  MIB,  describes  the  technique 
as  follows: 

"The  MIB  system  is  a  completely  automated  on-line  batch 
oriented  name  look-up  system  in  contrast  to  a  visual 
terminal  inquiry  response  system.    The  processing  of  in- 
quiries is  not  accomplished  through  use  of  an  algorithm 
such  as  a  Soundex  or  phonetic  technique;  instead,  a  name 
key  is  generated  (group  spelling  number)  through  use  of 
our  name  dictionaries.    These  dictionaries  of  surnames  and 
given  names  reside  on  a  disc  file.    The  dictionary  is  used 
only  to  convert  the  incoming  surname  or  given  name  to  a 
unique  group  number.    This  group  number  represents  names 
which  are  confused  with  one  another.    In  addition,  for 
given  names,  alternate  groups  are  generated  for  situations 
such  as  nicknames  and  other  confusions  which  do  not  neces- 
sarily sound  like  the  given  name  in  the  inquiry. 

In  contrast  to  other  popular  algorithms  which  attempt  to 
accomplish  the  above,  but  do  not  fully,  we  have  manually 
grouped  our  names  according  to  our  experience  over  the 
last  eight  years.    This  has  come  about  by  having  to  either 
search  or   update  at  least  160  million  records  over  this 
span  of  time.    Needless  to  say,  our  dictionaries  are 
fairly  complete  at  this  point  in  time. "20 

This  technique,  used  in  conjunction  with  a  .highly  specialized  search 
strategy  (discussed  later  in  this  section),   has  given  excellent  results. 
Mr.  Keighley  states  that: 

"In  order  to  give  you  some  measure  of  our  success,  we 
handle  all  the  situations  above  (compound  names,  nick- 
names, cross  reference  names  such  as  aliases  and  maiden 
names,  etc.)  against  a  file  of  14  million  records,  pro- 
cessing an  average  of  80,000  inquiries  a  day.    It  should, 
also,  be  pointed  out  that  no  manual  intervention,  either 
at  our  computer  site  or  at  our  subscribers  location  is 
needed  prior  to  conveying  the  results  of  our  search  to 
the  end  user,  since  we  identify  the  individual  with  a 
high  degree  of  accuracy. "^1 


Personal  Communication  from  Robert  Keighley,  Director  of  Systems  and 
Planninq,  Recording  and  Statistical  Division  of  Sperry  Rand,  the  servic- 
ing agent  for  the  MIB,  October  14,  1975. 


21 


Ibid. 
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This  example  illustrates  the  feasibility  of  developing  a  highly  success- 
ful retrieval  system  based  on  a  name  lookup  technique.    But  the  high  de- 
gree of  special  treatment  given  to  development  of  this  system  was  a  time- 
consuming  and  very  expensive  process.    It  is  equally  important  to  point 
out  that  successful  systems  based  on  modified  Soundex  rules  are  also 
available:    for  example,  the  retrieval  system  in  use  at  the  Department  of 
Justice  in  the  Immigration  and  Naturalization  Service.    In  other  words, 
although  the  MIB  system  represents  a  high  level  of  attainment,  there 
exist  algorithms  within  the  public  domain  which  can  be  used  very  effect- 
ively. 

A  fundamental  consideration  associated  with  the  adoption  of  any  technique 
is  of  course  the  tradeoff  between  computer  costs  for  implementing  and 
executing  complex  algorithms,  human  resources  for  monitoring  computer 
activities,  and  the  cost  of  missing  a  record  in  a  search,  which  may  have 
social  and  legal  implications.    In  view  of  the  latter,  it  seems  more  rea- 
sonable to  accept  larger  selectivity  ratios  based  on  name  lookup,  assum- 
ing that  other  non-unique  attributes  ca.n  be  effectively  used  to  filter 
lists  of  file  items  retrieved  by  surname  lookup. 


2.5    IMPLEMENTATION  OF  NAME  LOOKUP  TECHNIQUES 

The  high  selectivity  of  value  of  name  as  a  non-unique  identifier  has 
prompted  the  emphasis  on  name  lookup  techniques.    But  it  is  also  true 
that  this  emphasis  has  repercussions  regarding  selection  of  retrieval 
techniques.    Some  of  the  more  sophisticated  techniques  will  not  be 
easily  available  due  to  the  nature  of  possible  errors  in  the  data.  For 
example,  it  is  doubtful  that  hashing  techniques  would  be  useful  in  the 
processing  of  surnames  since  a_  misspelling  either  at  data  entry  or  re- 
trieval request  time  coul-d  result  in  a  complete  misdirection  of  the 
search. 

Use  of  name  lookup  techniques  facilitates  grouping  of  closely  related 
names.    Thus,  if  SMITH  and  SMYTHE  are  encoded  in  such  a  way  that  a  sort 
of  the  file  on  the  encoded  field  places  them  close  to  each  other,  the 
likelihood  of  finding  the  alternate  spelling  during  a  search  is  greatly 
enhanced.    For  example,  if  the  Soundex  code  appears  to  be  adequate  for 
handling  the  mix  of  names  in  the  file,  the  approach  taken  could  be  as 
follows:    apply  the  Soundex  algorithm  to  the  SURNAME  field  in  all  records, 
perform  a  sort  on  the  encoded  field,  and  write  the  records,  in  sort  order, 
on  a  storage  device.    The  records  will  be  recorded  in  convenient  logical 
groups  variously  called  bl ocks  or  buckets.    (Both  terms  imply  the  number 
of  physical  records  involved  in  a  single  read  or  write  operation.) 

If  the  procedure  described  above  were  followed,  it  would  be  likely  that 
SMITH  and  SMYTHE  would  be  recorded  in  the  same,  or  at  least  in  adjacent, 
buckets.    Indices  would  then  be  formed  for  use  in  locating  buckets  during 
a  search.    In  a  small  file,  there  might  be  a  single  level  of  index  based, 
for  example,  on  the  Soundex  formulation  of  the  input  surname,     ihe  first 
two  or  three  alphanumerics  of  the  resulting  code  could  point  to  a  bucket 
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that  would  be  expected  to  contain  the  desired  record.    In  a  larger 
system,  a  multi-level  index  scheme  might  be  used  with  the  highest  level 
pointing  to  a  secondary  index  containing  a  breakdown  of  names  starting 
with  the  letter  'S.'    In  all  cases,  the  indicated  bucket  will  be  read 

:   in  and  a  matching  algorithm  followed  until  either  an  exact  match  is 
found  or  the  contents  of  the  bucket  have  been  exhausted.    If  no  match 
is  found,  the  search  may  continue  through  use  of  the  preceding  and  sub- 
sequent buckets.    In  some  cases,  it  may  be  reasonable  to  look  in  logical 
blocks  that  are  two  or  three  buckets  away  since  in  the  case  of  very 

j  common  names  the  instances  may  stretch  over  several  buckets. 

j  The  system  developed  by  the  Immigration  and  Naturalization  Service  fol- 
lows this  type  of  approach  with  the  Soundex  codes  producing  the  top 
level  index.    The  resulting  groupings  are  then  further  segregated  on 
the  basis  of  the  first  names.    The  third  level  of  index,  BIRTHDATE, 

,1  gives  flexibility  to  the  retrieval  system.    The  program  is  capable  of 

i   handling  requests  for  the  records  of  individuals  of  exactly  this  age  or 
for  the  records  of  those  within  bounds  of  plus  or  minus  nine  years.  The 
user  has  the  option  of  asking  for  a  count  of  hits  before  results  are 
printed  in  case  an  unmanageable  number  of  records  have  been  retrieved 

j  due  to  specification  of  a  wide  age  range. 22a    obviously  this  option  is  for 
use  by  inquirers  who  are  not  sure  of  the  individual's  exact  age  and  not 
by  the  individual  requesting  his  own  record.    However,  the  number  of 
problems  than  can  occur  with  the  AGE/BIRTHDATE  variable  make  it  not  un- 
likely that  an  individual  could  be  mistaken  about  how  his  own  age  has 
been  recorded.    The  handling  of  such  an  error  could  be  simplified  by 

j  the  availability  of  a  range  field. 

The  search  mechanism  employed  by  the  Immigration  and  Naturalization 
Service  necessitates  use  of  three  levels  of  indices  before  the  actual 
accessing  of  the  record(s).    The  number  of  required  index  or  file 
searches  can  be  reduced  through  use  of  encoding  techniques  to  form  a 
compound  pointer.    A  clever  approach  to  using  several  attributes  as  a 
record-key  index  or  compound  pointer  was  developed  for  the  Central 
California  Red  Cross  Blood  Center.    The  program,  whose  main  purpose  is 
to  maintain  and  develop  a  donor  pool  of  repeat  donors,  was  in  operation 
at  Stanford  University  at  the  time  of  this  study. 

"Donor  records  are... stored  in  contiguous  blocks  by  blood 
type,  30/block,  and  the  donor  indexes  that  point  to  these 
records  are  stored  255/block.    By  this  method  of  indexing, 
the  indexes  themselves  can  be  sorted  by  blood  type,  date 
of  last  donation,  etc.  relatively  easily  so  that  one  set 
of  donors  can  be  dealt  with  individual ly. "22b 

i^Zapersonal  Communication  from  Robert  Robinson,  Director  of  ADP  Systems 
Branch,  Immigration  and  Naturalization  Service,  Department  of  Justice, 
September  22,  1975. 

22bLudwig,  H.  R.,  "An  Interactive  Computer-Based  Donor  Management  System," 
Computers  in  Biology  and  Medicine,  5  (1975),  pp.  69-75. 


27 


The  donor  index  contains  the  following  attributes: 

-  a  two-byte  donor  i.d.  consisting  of  the  first  three  let- 
ters of  the  last  name  and  the  birthdate.    This  informa- 
tion is  reduced  to  two  bytes  through  use  of  the  following 
algorithm: 

3 

Key  =  E  (a+26^"'')+0-2400 
i  =  l 

where  a^.  denotes  the  numerical  value  of  each  of  the  first 
three  letters  of  the  last  name,  e.g.  A  =  1,B  =  2, ...and 
J  denotes  the  Julian  value  of  the  birthdate  with  the  base 

beim  1900. 

-  a  two-byte  pointer  that  specifies  both  the  physical  block 
and  the  logical  record  number.    For  the  current  file, 
this  is  a  number  between  1  and  29,940.    The  organization 
of  the  file  is  such  that  this  number  also  implies  blood 
type . 

-  a  two-byte  data  field  to  indicate  date  of  last  donation. 
To  reduce  field  size,  this  is  a  Julian  date  from  some 
base  year  (in  this  case,  1972). 

-  two  one-byte  fields  for  coded  information:    number  of 
entries  on  the  elioibility  list,  total  donations  to 
date,  indication  of  rare  blood  types,  etc. 

The  system  has  been  designed  so  that  updating  of  some  levels  of  infor- 
mation can  be  handled  by  reading  in  only  the  index  level,  and,  when  a 
donor  record  is  required,  it  can  be  fetched  directly. 

A  very  sophisticated  example  of  use  of  an  encoding  technique  occurs  in 
the  MIB  system.    This  system  places  a  high  premium  on  accuracy  of 
retrieval.    Dictionary  lookups  are  performed  to  obtain  group  numbers 
both  for  surname  and  given  name.    Mr.  Keighley  has  summarized  the  search 
algorithm  as  follows: 

"The  process  of  looking  up  an  individual  using  surname,  given 
name,  date  of  birth,  place  of  birth  (optional)  and  territory 
of  residence  (optional)  is  as  follows: 

1.  Look  up  surname  in  surname  dictionary  to  obtain  its  group 
number. 

2.  Look  up  given  name  in  given  name  dictionary  to  obtain  its 
group  number  and  any  alternatives  (nicknames  etc.). 
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3.  Generate  a  key  consisting  of  surname  grouo  number,  given 
name  group  number,  date  of  birth  (coded),  place  of  birth 
(optional-coded),  and  territory  (optional -coded) . 

4.  Using  the  generated  key  from  Step  3,  search  the  data  base 
on  a  direct  access  basis  (index  sequential)  for  records 
applicable  to  the  inquiry. 

5.  Screen  each  record  in  detail  and  mathematically  determine 
what  the  degree  of  probability  is  that  the  record  being 
screened  pertains  to  the  person  being  searched. 

6.  If  any  record  being  screened  has  a  high  enough  proba- 
bility, it  is  selected  as  a  response  to  the  inquiry. 

In  this  instance  the  encoding  technique  has  not  cut  down  on  the  number 
of  required  accesses,  but  has  instead  been  used  to  generate  a  very 
accurate  filter. 


2.6    OTHER  SEARCH  STRATEGIES 

Analysts  considering  potential  system  changes  will  do  so  in  the  context 
of  the  operational  environment  of  their  agencies.    In  some  instances, 
no  great  emphasis  will  be  placed  on  search  strategy.    Yet,  in  other 
situatipns,  particularly  when  the  file  size  is  large,  the  selection  of 
an  efficient  search  strategy  will  be  crucial  to  effective  system  per- 
formance.   The  number  of  requests  to  be  processed,  batch/interactive 
capabilities  of  the  system,  size  of  the  file,  etc.,  will  be  determining 
factors  in  these  decisions.    A  general  discussion  of  some  available 
search  strategies  is  given  here  as  background  for  those  who  must  concern 
themselves  with  this  issue. 

W.A.  Burkhard  and  R.M.  Keller  have  described  an  approach  to  the  problem 
of  "...searching  the  set  of  keys  in  a  file  to  find  a  key  which  is 
closest  to  a  given  query  key...  Three  file  structures  are  presented  to- 
gether with  their  corresponding  search  algorithms,  which  are  intended  to 
reduce  the  number  of  comparisons    required  to  achieve  the  desired  re- 
sult. '"^'^   They  describe  their  result  as  follows: 

"The  study  described  here  is  concerned  with  the  general 
problem  of  efficiently  searching  files  in  the  following 
situations:  (1)  find  a  key  in  the  file  closest  to  a 
given  query  key,  and  (2)  find  all  kevs  in  the  file 


Keighley,  op. ci t. 

W.  A.  Burkhard  and  R.  M.  Keller,  "Some  Approaches  to  Best-Match  File 
Searching,"  Communications  of  the  ACM,  16,  (4  April  1973),  p.  230. 
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closest  to  a  given  query  key.    By  'closest'  we  mean  ac- 
cording to  some  suitable  measure  of  distance,  specifically 
a  'metric'  in  the  mathematical  sense.    These  situations  may 
arise  in  information  retrieval  applications  in  which  mem- 
bers of  a  file  are  keyed  to  a  number  of  numerically-re- 
presented attributes.    Our  search  algorithms  have  the 
following  common  basis:    let  b  be  the  'best'  key  found  at 
a  certain  point  in  the  application  of  the  algorithm. 
Based  on  certain  'cutoff  criteria'  which  depend  on  b,  sub- 
sets of  tne  file  remaining  to  be  examined  are  eliminated 
without  explicitly  comparina  the  keys  in  these  subsets 
with  the  query  key.    Our  experimental  results  (on  randomly- 
generated  files)  indicate  that  large  segments  of  the  file 
may  be  eliminated  from  consideration  using  these  cutoff 
criteria. "25 

Abraham  Bookstein  has  described  a  hybrid  search  technique  which 
"...has  the  advantage  of  retaining  the  simplicity  of  search  keys  while 
also  including  some  of  the  flexibility  that  Boolean  expressions  of  key 
words  have  for  uniquely  defining  an  item,     ...The  only  indexes  that 
must  be  maintained  are  the  hash  tables;  the  other  indexes ...  are  replaced 
by  the  search  algorithms. "26    Bookstein  goes  on  to  say: 

"A  user  would  begin  by  entering  into  the  system  search  key. 
If  the  system  finds  that  the  number  of  items  that  would  be 
retrieved  exceeds  a  preset  threshold,  it  would  output  a  mes- 
sage requesting  that  the  user  enter  a  set  of  key  words  taken 
from  various  fields  in  the  records;  ...The  system  first  genera- 
tes a  subfile  of  records  having  the  desired  search  key.  If 
a  hashing  technique  is  used,  constructing  this  subfile  can  be 
accomplished  quickly  and  at  a  relatively  little  cost  in  space 
for  tables.    Once  the  smaller  file  is  formed,  a  complete 
search  of  the  full  records  can  be  made  for  the  key  words. 
Since  the  system  operates  in  two  phases,  it  is  less  sensitive 
to  the  number  of  records  the  search  key  retrieves  as  far  as 
user  considerations  are  concerned.    Ease  of  use  becomes  the 
dominating  objective  in  designing  the  search  key  ...  In  the  hy- 
brid system,  we  can  think  of  the  search  key  not  as  an  access 
mechanism  ...  but  rather  as  a  file  reduction  mechanism.  This 
system  trades  the  cost  of  maintaining  and  storing  large  indexes 
for  an  increase  in  costs  of  computer  processing;  only  relatively 
easily  maintained  hash  tables  for  fixed  length  search  keys  need 
to  be  maintained. "27 


^-"Ibid. ,  p.  236. 

^^Abraham  Bookstein,  "A  Hybrid  Access  Method  for  Bibliographic  Records," 
Journal  of  Library  Automation,  7:97-104,  (2  June  1974),  p.  99. 

^^Ibid. 
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The  use  of  Malcolm  Harrison's  hash  technique  to  expedite  the  search  is 
explained  as  follows: 

"A  fixed  number  of  bits,  or  signatures,  (is)  added  to  each 
field  on  which  a  search  can  take  place;  these  additional 
bits  are  derived  in  a  well-defined  way  from  the  original 
field.    This  subfield  is  a  fixed-size  representation  of  the 
full  field  in  a  form  that  can  be  used  to  very  rapidly  elimi- 
nate most  records  which  would  not  pass  the  key  word  matching 
test.    It  is  stored  in  the  index  to  the  file  along  with  the 
address  of  the  record.    Though  this  Dreliminary  test  is  not 
foolproof,  it  could  considerably  reduce  the  size  of  the  sub- 
file that  requires  a  more  costly  complete  search,  thereby 
reducing  the  number  of  disc  accesses. "2° 

The  author  quotes  a  false  drop  rate  of  approximately  ten  percent  for  the 
bibliographic  application  being  described.    This  is  of  course  too  high 
a  rate  for  personal  data  file  retrieval  systems,  yet  a  careful  implementa- 
tion of  a  similar  combination  of  such  techniques  might  be  appropriately 
applied  to  the  retrieval  problem  at  hand. 

Retrieval  techniques  described  in  this  section  were  selected  because  of 
their  relevance  to  the  present  discussion  of  use  of  non-unique  identifiers 
in  search  algorithms.    Those  interested  in  pursuing  a  broader  range  of 
ij  retrieval  techniques  might  begin  with  Dennis  Severance's  generalized 
J  mode]  for  exploring  the  relationship  between  alternative  search  technl- 
;  ques^^  and  Knuth's  work  on  sorting  and  searching30. 


29 

I     Dennis  G.  Severance,  "Identifier  Search  Mechanisms:    A  Survey  and 

i      Generalized  Model,"  Computing  Surveys,  6  (3  September  1974),  pp.  175-194. 


Knuth,  op.ci t. 
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3.0    IDENTIFICATION  OF  WEIGHTING  FACTORS 


3.1    ACCURACY  CONSTRAINTS  IMPOSED  BY  THE  PRIVACY  ACT 

One  of  the  most  important  facets  of  the  privacy  legislation  is  the 
imposition  of  accuracy  constraints  on  the  agencies  maintaining  personal 
data  files.    The  intent  of  these  provisions  is  to  protect  the  individual's 
privacy  by  insisting  that  data  pertaining  to  him  be  correct,  up-to-date, 
and  relevant.    The  new  law  gives  the  individual  the  right  to  review  his 
own  record  and  to  request  that  any  relevant  corrections  be  made.  Disputed 
information  maintained  in  the  file  must  be  so  identified. 

This  emphasis  on  accuracy  of  personal  data  held  in  computer  files  is  a 
function  of  data  availability,  not  of  an  increase  in  error  rates  due  to 
computerization  of  the  data.    Manual  record-keeping  systems  have,  in 
general,  a  higher  error  rate  than  computer  systems.    Susan  Wooldridge, 
et  aT  have  stated  that  an  error  rate  of  up  to  10%  is  not  unusual  in 
paper-based  files.    "Indeed  a  frequently  used  justification  for  computer- 
izing is  to  raise  the  accuracy  of  the  files. "31 

Alan  Westin  has  quoted  Dr.  Collen  of  Kaiser-Permanente  as  indicating  that 
manual  systems  in  the  hospital  setting  are  plagued  by  errors. 

"Errors  have  become  so  common  in  traditional  medical  records 
that  physicians  are  accustomed  to  handling  them,  from  finding 
a  woman's  lab  test  in  her  husband's  file,  to  prescription 
errors  in  the  hospital. "32 

Dr.  Collen  went  on  to  say  that,  after  installation  of  a  computerized 
record-keeping  system,  they  had  found  that: 

"...some  10  percent  of  our  computer  records  contain  'errors.' 
We  don't  know  what  the  error  level  in  manual  files  is,  but 
it  certainly  couldn't  be  less.    Gradually  we  hope  to  get 
the  computer  file  error  level  down  to  around  three  percent. 
To  get  to  0  percent  would  be  prohibitively  costly,  and 
probably  isn't  possible,  anyway. "33 

Before  the  emergence  of  computerized  files,  few  organizations  took  time 
to  study  their  own  systems  from  the  point  of  view  of  accuracy  and  possible 
means  of  improvement. 34    There  was  far  less  concern  with  the  problem,  to  a 


Wooldridge,  Susan,  Colin  Corder,  and  Claude  Johnson,  Securi ty  Standards 
for  Data  Processing,  John  Wiley  and  Sons,  New  York,  1973,  p.  60. 

32 

Westin,  A.,  Databanks  in  a  Free  Society ..  Quadrangle  Books ,  1972^  p.  210. 
•'''ibid. 


Ibid,  p.  433. 
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great  extent  because  clues  other  than  the  data  itself  were  readily  avail- 
able.   These  clues  took  the  form  of  the  handwriting  in  which  data  was 
recorded,  the  color  of  the  form  used,  the  placement  of  the  paper  in  the 
file,  the  age  of  the  document,  etc.    In  many  cases,  the  person  reviewing 
the  data  was  conversant  with  the  general  structure  of  the  file,  and  often 
even  with  the  individual  case.    He  thus  brought  his  own  background  to  the 
perusal  of  the  material. 

Automation  has  had  the  effect  of  depersonalizing  the  data,  reducing  it 
artificially  to  a  level  of  uniformity  that  masks  the  disparities  in 
accuracy  while  at  the  same  time  making  it  readily  available  to  a  broad 
spectrum  of  users.    The  loss  of  visual  clues  such  as  ancient  appearance 
of  the  document  or  illegibility  of  the  data  is  particularly  important 
from  the  point  of  view  of  timeliness  of  the  data  maintained  in  the  file. 

James  Martin,  in  his  book  entitled  Security,  Accuracy,  and  Privacy  in 
Computer  Systems,  has  discussed  this  problem. 

"In  addition  to  patent  errors,  large  files  also  accumulate  two 
other  kinds  of  troublesome  material --the  decaying  and  the  un- 
evaluated.    The  former  is  characteristic  of  biographical  details. 
Information  gets  less  reliable  the  farther  away  it  is  from  the 
source.    References  and  testimonials,  to  say  nothing  of  college 
degrees,  lose  their  relevance  as  their  subject  grows  older.  The 
facts  remain  facts  in  the  environment  of  their  collection,  but 
they  may  be  inappropriate  in  the  context  of  the  use  to  which 
they  are  put. "35 

The  decay  factor  inherent  in  much  biographical  data  might  be  thought  of 
as  the  length  of  time  during  which  the  data  is  more  likely  to  be  right 
than  wrong.    For  some  variables,  such  as  birthplace,  there  is  no  decay. 
For  others,  such  as  current  age,  decay  is  at  a  constant  rate.    For  still 
others,  the  amount  of  decay  will  depend  upon  the  time  at  which  the  data 
was  collected,  e.g.,  the  variable  HIGHEST  DEGREE  ATTAINED  will  have  a 
much  greater  potential  for  change  if  it  refers  to  a  student  than  if  it 
refers  to  a  professional  person. 

Form  3.1-1  identifies  some  of  the  standard  biographical  data  items  that 
are  subject  to  this  type  of  variability.    The  privacy  legislation  has 
emphasized  the  importance  of  the  timeliness  factor  since  clearly  much  of 
this  data  deteriorates  over  time.    In  the  context  of  the  current  discus- 
sion, this  factor  is  an  important  indicator  in  the  assignment  of  weighting 
scores  to  be  used  in  a  final  determination  of  selection  of  non-unique 
identifiers  for  retrieval  algorithms. 


Martin,  J.,  Security,  Accuracy,  and  Privacy  in  Computer  Systems, 
Prentice-Hall,  New  Jersey,  1973,  p.  431. 
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Form  3.1-1    Decay  Factors  of  Standard  Biographical  Date  Elements 


NON-UNIQUE  IDENTIFIER 

DECAY  FACTOR 
Does  this  variable  remain  constant 
over  time? 

YES* 

NO** 

LAST  NAME 

MALE 

/ 

FEMALE 

.  

/ 

First  Name 

/ 

Middle  Name 

/ 

Sex 

/ 

Birthdate 

/ 

Birthplace 

/ 

Marital  Status 

/ 

Address 

/ 

Occupation 

/ 

Education 

/ 

Physical  Characteristics 

/ 

*Changes  are  rare 
■*Changes  are  not  unusual 


I 
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3.2    DETERMINATION  OF  WEIGHTING  FACTORS  FOR  EXISTING  FILES 


In  the  establishment  of  a  computer  file,  errors  can  be  introduced  at  any 
stage  of  data  handling.    They  sometimes  result  from  the  individual's 
misunderstanding  of  a  question,  sometimes  it  is  a  transcription  error  on 
the  part  of  the  clerk  filling  out  a  keypunch  form,  sometimes  it  is  a 
problem  in  the  file  management  system  that  creates  or  updates  the  record. 
A  determination  of  the  likelihood  of  such  errors  having  occurr-^d  requires 
evaluation  of  procedures  followed  during  the  key  stages  in  data  handling: 

1.  data  base  definition  and  forms  design 

2.  data  collection  techniques 

3.  keying  of  data  and  input  controls 

4.  computer  edit  routines 

5.  manual  accuracy  check  of  record  as  printed  by  the  computer 

Thus,  an  acquaintance  with  the  evolutionary  process  through  which  the 
file  came  into  existence  is  essential  to  an  evaluation  of  the  degree  of 
accuracy  of  the  data  it  contains.    A  comprehensive  examination  of  this 
evolutionary  process  should  answer  such  questions  as:    is  the  data 
contained  in  the  files  consistent?    (i.e.,  did  all  of  the  individuals 
who  filled  out  the  forms  understand  the  same  thing  by  the  questions? 
Was  the  question  asked  the  same  way  each  time  or  has  the  wording  changed 
significantly?)    and:    which  data  items  in  the  file  are  likely  to  be 
most  reliable? 

The  following  sections  describe  each  stage  of  data  collection  in  terms  of 
appropriate  error  detection  procedures.    Acknowledgement  of  the  possible 
existence  of  such  errors  is  important,  but,  since  data  base  deficiencies 
will  differ  greatly  from  system  to  system,  it  would  be  helpful  to  be  able 
to  characterize  these  weaknesses  in  more  specific  terms.    One  approach 
would  be  to  assign  weighting  scores  to  the  various  biographical  identifiers 
in  the  file  in  such  a  way  that  the  weights  serve  as  indicators  of  the 
adequacy  of  the  collection  procedures  for  each  variable.    In  the  following 
discussions,  a  vehicle  for  making  such  a  determination  has  been  suggested. 
Included  with  the  description  of  each  stage  of  data  collection  is  a  table 
in  which  YES/NO  questions  are  postulated  for  each  of  the  problem  areas 
identified.    These  questions  are  to  be  asked  for  each  of  the  biographical 
identifiers  used  in  the  hypothetical  file  under  consideration.    A  summary 
chart  is  used  to  record  the  total  number  of  NO  checks  for  each  of  the  data 
elements.    These  totals  then  become  the  identified  weighting  scores  for 
the  various  data  items  and  thus  produce  a  gross  picture  of  the  validity  of 
the  biographical  identifiers  in  the  file. 

The  weights  thus  arrived  at  for  each  of  the  standard  biographical  data 
elements  will  be  used  in  a  final  determination  of  identifiers  for  the 
retrieval  process  (see  Section  4). 


35 


3.2.1    Data  Base  Definition  and  Forms  Design.    The  determination  of  an 
appropriate  set  of  data  items  to  be  included  in  a  system  is  an  important 
part  of  the  design.    If  the  items  requested  are  inappropriate  or  the 
wording  of  the  question  obscure,  difficulties  arise  which  are  not  obvious 
from  looking  at  the  data.    The  importance— and  the  difficulty--of  obtain- 
ing such  uniformity  of  data  is  discussed  in  the  Uniform  Hospital  Discharge 
Data  Demonstration  report.    This  project  was  directed  toward  improvement 
in  health  care  delivery  through  collection  and  evaluation  of  discharge 
data  from  a  large  number  of  hospital  facilities  throughout  the  country. 
A  subcommittee  of  the  U.S.  National  Committee  on  Vital  and  Health 
Statistics  handled  selection  of  the  basic  data  set.    Collection  of  the 
data  was  carried  out  systematically,  with  personnel  at  the  hospital  sites 
being  trained  with  respect  to  abstracting  of  data  items  from  the  charts, 
and  quality  control  checks  implemented  tnroughout  the  data  collection 
procedure. 

"In  all  instances,  the  system  manuals  used  in  the  test  hospitals 
were  modified  to  conform  the  UHDDD  manual ...  Coding  examples  were 
provided  to  elaborate  on  the  individual  items  and  to  cover 
anticipated  contingencies  such  as  when  the  date  of  birth,  ZIP 
code,  or  married  status  was  unknown,  when  patients  from  foreign 
countries  who  do  not  have  ZIP  codes  were  treated,  etc.  Examples 
to  clarify  the  concepts  of  the  principal  diagnosis  and  the 
principal  procedure  were  also  given. "^^ 

Even  so,  many  data  collection  problems  have  occurred.    For  example,  the 
categories  established  for  MARITAL  STATUS  are: 

"Now  Married  refers  to  persons  who  state  they  are  married  and 
not  separated.  Divorced  or  Separated  is  used  for  persons  who 
state  they  no  longer  live  together,  whether  or  not  legal  action 
has  been  taken.  Never  Married  (Single)  refers  to  persons  who 
state  they  have  never  married  or  whose  only  marriage  has  been 
annulled.    Widowed  includes  widows  and  widowers. "3' 


Yet, as  noted  in  Section  2.2,  there  were  difficulties  in  collecting  even 
this  very  well-defined  data  in  the  following  instances: 

"Persons  living  in  common-law  relationships  who  do  not  wish  to 
be  classified  under  any  of  the  alternatives.    Divorced  persons 
who  consider  themselves  single  and  state  so  at  admission. 
Single  females  admitted  for  childbirth  do  not  generally  want 
to  be  identified  as  single  and  state  they  are  married. "^8 


Hodgson,  D.  A.,  L.  E.  Kucken,  and  J.  M.  Ensign,  The  Uniform  Hospital 
Discharge  Data  Demonstration,  Surrmary  Report,  Health  Services  Founda- 
tion, Chicago,  Illinois,  1973,  p.  24. 

-^^Ibid,  p.  12. 

^^Ibid,  p.  31. 
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They  conclude: 

"The  original . intent  of  this  item  was  to  attempt  to  secure  some 
indication  of  the  living  arrangement  of  a  patient,  but  it  proves 
to  be  of  marginal  utility  for  this  purpose.    This  is  a  complex 
variable  to  establish  if  the  many  socio-economic  factors  which 
enter  into  the  underlying  health  care  implications  associated 
with  living  arrangements  are  consi dered. "39 

Form  3.2.1-1  contains  an  analysis  of  the  inherent  characteristics  of  a 
sample  set  of  biographical  data  elements.    The  results  indicated  in  the 
form  reflect  good  experience,  but  it  is  assumed  that  counter  instances 
could  be  found.    The  analyst  should  compare  his  own  evaluation  of  similar 
elements  contained  on  the  forms  used  in  his  data  collection  procedures 
with  these  results. 

The  data  requested  in  Form  3.2.1-2,  unlike  that  shown  in  the  earlier 
forms,  is  dependent  upon  the  actual  collection  procedures  involved  and 
is,  therefore,  left  empty.    The  analyst  is  encouraged  to  produce  similar 
tables,  using  the  identifiers  relevant  to  his  file,  and  then,  using  a 
selection  of  random  records  from  the  data  base  in  conjunction  with  the 
forms  on  which  the  original  data  collection  took  place,  to  answer  the 
questions  in  the  table.    The  answers  will  supply  the  required  weights 
for  this  stage  of  the  data  collection  procedure. 

3.2.2    Data  Collection  Techniques.    Perhaps  the  best  possible  guideline 
for  successful  data  collection  would  be:    capture  the  data  as  close  to 
the  source  as  possible.    This  might  equally  well  be  stated  as:    the  fewer 
manual  transcriptions  of  the  data,  the  cleaner  the  data.    The  reasoning 
behind  this  is  that  the  individual  is  normally  the  person  best  qualified 
to  monitor  collection  of  information  about  himself.    He  would  not  be  as 
likely,  for  example,  to  transpose  numbers  in  his  street  address  as  would 
a  clerk  whose  job  is  to  transfer  his  descriptive  data  onto  a  keypunch 
form.    Assuming  this  is  a  reasonable  generalization,  an  ideal  data  col- 
lection system  would  then  call  for  the  individual  to  enter  his  data 
directly  onto  a  keypunch  form  (which  need  not  look  like  the  standard  80 
column  form,  but  which  must  be  marked  with  column  numbers  that  are  re- 
cognizable to  the  keypunch  staff),  or,  in  some  cases,  for  the  individual 
to  interact  with  the  computer,  via  terminal,  in  a  question/answer  dia- 
logue.   If  the  questions  are  easily  understood,  if  they  are  available  in 
the  language  spoken  by  the  individual,  and  if  he  prints  (or  types,  if  a 
terminal  is  used)  acceptably  well,  this  may  indeed  be  the  best  way  to 
capture  the  data. 
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The  problem  is  usually  not  so  straightforward,  however.    If  any  ambiguity 
exists  in  the  wording  of  the  question,  or  if  coding  of  the  data  is  re- 
quired, it  may  be  far  more  relevant  to  have  the  forms  filled  out  by 
trained  interviewers  who  understand  the  intent  of  the  questions  and  can 
guide  the  individual  in  determining  what  is  for  him,  the  appropriate 
answer. 


These  generalities  concern  data  collected  specifically  for  entry  into 
the  computer.    There  is   another  set  of  instances  which  involve  collec- 
tion of  data  after  the  fact,  as  in  the  case  in  the  Uniform  Hospital 
Discharge  Data  Demonstration  summary.    Here  existing  medical  records  are 
being  assigned  to  the  abstracting  task.    The  methodology  for  this 
abstracting  procedure  has  been  carefully  laid  out.    Even  so,  the  accuracy 
checks  carried  out  by  means  of  a  re-abstracting  procedure  for  the  data 
collected  in  Months  1  and  4  at  each  hospital  in  some  cases  reflect  a 
surprisingly  high  error  rate. 

In  the  following  paragraphs,  the  re-abstracting  procedure  is  quoted  as 
it  appears  in  the  project  manual,  along  with  the  summary  rates  of  errors 
that  occurred. 

"The  demonstration  design  called  for  two  samples  of  medical 
records  to  be  re-abstracted  in  each  test  site.    The  first  re- 
abstracting  was  designed  to  compare  the  data  produced  by 
applying  the  UHDDD  definitions  to  the  data  generated  from 
the  same  medical  records  using  the  definitions  in  effect 
prior  to  the  demonstration.    The  second  re-abstracting  com- 
pared the  data  produced  by  the  hospital  during  the  fourth 
month  of  the  demonstration,  with  the  data  re-abstracted  by 
the  project  medical  record  librarians. . .Table  2  provides  a 
summary  of  the  discrepancy  rates  for  individual  items  from 
the  second  re-abstracting  analyses.    In  almost  all  instances, 
the  errors  with  items  having  low  discrepancy  rates  were  the 
result  of  clerical  errors;  e.g.,  an  error  in  transcribing 
data  from  the  medical  record  to  the  abstract. 
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Table  2    Percentage  of  Abstracts  with 
Discrepancies  by  Test  Sites 


ITEM 

MAINE 

PENN. 

WIS. 

CALIF. 

PERSON  I.D. 

0.1 

0.6 

NA* 

0.1 

DATE  OF  BIRTH 

0.3 

1.2 



0.7 

SEX 

0.8 

0.6 

1.2 

0.5 

MARITAL  STATUS 

1.0 

0.8 

1.8 

1.1 

RACE 

0.2 

0.4 

1.0 

0.5 

ZIP  CODE 

— 

1.1 

2.6 

0.5 

ADMISSION  DATE 

0.1 

— 

0.3 

— 

ADMISSION  HOUR 

1.3 

24.0 

15.5 

2.0 

DISCHARGE  DATE 

0.1 

0.2 

NA 

— 

ATTENDING  PHYSICIAN 

0.4 

1.4 

11.6 

4.1 

OPERATING  PHYSICIAN 

0.1 

2.4 

19.7 

NA 

PRINCIPAL  DIAGNOSES 

8.3 

11.8 

10.7 

NA 

OTHER  DIAGNOSIS 

14.6 

21.0 

30.5 

NA 

PRINCIPAL  PROCEDURE 

1.9 

6.5 

22.4 

NA 

OTHER  PROCEDURES 

1.5 

9.3 

15.8 

NA 

DATES  OF  PRINCIPAL  AND 

OTHER  PROCEDURES 

0.1 

7.4 

18.2 

SERVICE  TO  WHICH  ADMITTED 

2.6 

3.2 

3.7 

2.5 

DISPOSITION  OF  PATIENT 

0.4 

1.2 

3.9 

0.4 

PRINCIPAL  SOURCE  OF  PAYMENT 

3.7 

2.2 

1.6 

2.7  40 

*Not  Available 

Obviously  the  principal  difficulty  in  this  data  collection  procedure  is 
in  the  training  of  clerical  staff  charged  with  properly  interpreting  the 
available  data.    C.  R.  Newing,  in  discussing  data  collection  problems  in 
the  insurance  business,  points  up  the  necessity  of  checking  data  collec- 
tion procedures  and  tracing  errors  back  to  the  point  of  origin. 

"In  our  efforts  to  control  quality,  training  of  clerical  staff 
plays  a  large  part  particularly  as  most  of  the  data  sheets 
which  we  use  are  quite  complicated  and  contain  a  large  amount 
of  information  in  coded  form.    We  have  found  it  worth  while  to 
return  documents  which  contain  errors  to  the  originator  and 
while  this  is  not  popular  it  does  seem  to  be  having  effect. 
There  was  a  time  when  20  percent  or  so  of  documents  contained 
errors  but  we  have  reduced  this  to  around  5  percent...  Having 
adopted  the  principle  that  the  files  which  we  hold  are  the 
responsibility  of  the  user  departments  as  far  as  content  and 
accuracy  are  concerned,  we  co-operate  in  helping  them  to  apply 
any  checks  which  they  may  wish  to  operate. "41 


Ibid-,  p.  38. 

Newing,  C.  R. ,  "Data  Arrangement  and  Checking  in  the  Insurance 
Business,"  Applied  Statistics,  Vol.  2,  No.  1,  1972,  pp.  45-52. 

41 


In  anotlier  paper  on  a  similar  subject,  W.  F.  Kertisley  emphasized  the 
detailed  analysis  required  for  data  that  is  initially  "crude"  in  form. 
He  reported  on  a  survey  directed  at  collection  of  "...statistical  data 
on  monetary  transactions  in  the  personal  section."    Data  collection 
procedures  called  for  family  interviews  as  well  as  preparation  of  a  14- 
day  diary  by  the  family.    Many  problems  were  encountered  by  the  editors. 

"Some  deficiencies  become  obvious  as  soon  as  the  entries  are 
examined;  others  come  to  light  when  data  in  one  part  of  the 
budget  are  compared  with  another  part.    Some  inconsistencies 
are  found  in  comparisons  between  diary  entries  and  interview 
data;  others  when  interview  data  are  compared  with  individual 
or  family  circumstances. "42 

In  general,  he  felt  that  the  necessity  for  making  editing  changes  had 
resulted  from: 


"(a)  duplicated  information; 

(b)  difficulties  in  applying  definitions; 

(c)  incomplete  or  inconsistent  information; 

(d)  timing  of  transactions; 

(e)  changes  in  household  composition."'*^ 

Form  3.2.2-1  contains  a  set  of  questions  relevant  to  the  methodology  of 
data  collection  for  each  item.    If  the  information  contained  in  the  file 
was  handled  differently  at  different  stages  of  evolution,  the  analyst  may 
want  to  fill  out  two  sets  of  tables,  or  he  may  choose  to  use  the  latest 
methodology.    If  he  does  this,  he  must  remember  that  his  final  results 
do  not  fully  reflect  the  status  of  the  current  file. 


3.2.3    Keying  of  Data  and  Input  Controls.    The  entry  of  data  into  a  com- 
puter file  is  accomplished  in  one  of  two  ways:    interactively,  via 
terminal  operating  in  a  real  time  environment,  or  batch,  with  data  being 
keyed  ahead  of  time  to  cards,  tape,  disk,  etc.    Error  rates  for  these 
methods  tend  to  be  in  the  same  general  range  (from  1-4  percent).  Con- 
sequently, it  is  the  degree  of  control  placed  on  the  overall  handling 
of  the  data  by  the  system  that  makes  the  difference,  and  not  the  choice 
of  keying  device.    Entry  of  data  via  terminal,  however,  creates  an 
entirely  different  set  of  problems.    In  this  case,  control  is  a  function 
of  system  efficiency  and  error  rates  will  depend  on  the  degree  of 
sophistication  of  the  programs  themselves.    James  Martin  has  discussed 
this  difference  in  approach: 


^^Kemsley,  W.  F. ,  "Pre-Computer  Editing  of  Budgets  for  the  Family  Expen- 
diture Survey,"  Applied  Statistics.  Vol.  2,  No.  1,  1972,  pp.  58-64. 

Ibid. 
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"In  some  systems  a11  information  stored  originates  from  the 
terminal  operators.    Sometimes  the  terminal  operators  work  in 
a  fairly  casual  manner,  compared  with  the  card-punch  operators 
and  verifiers  of  batch  data  processing.    Clearly  the  viability 
of  the  concept  of  building  up  a  data  base  in  this  way  depends 
upon  whether  we  can  control  the  accuracy  of  the  terminal  input, 
catching  the  errors  that  occur. 

"On  certain  systems  the  errors  are  cumulative.    The  files  con- 
tain information  about  a  set  of  items  that  is  kept  for  several 
months  and  is  updated  periodically  by  terminal  operator  actions. 
If  occasional  operator  actions  cause  errors  in  the  files,  as 
the  months  pass  by  the  files  will  steadily  collect  more  and 
more  inaccuracies.    This  situation  could  clearly  bring  the 
system  into  disrepute,  and  controls  must  be  devised  to  prevent 
it. 

"A  number  of  factors  make  real  time  systems  worse  than  batch 
systems  for  the  control  of  accuracy.    First,  there  are  likely 
to  be  more  terminal  operators  creating  the  input  and  they  are 
scattered  over  many  areas.    They  tend  to  be  more  diverse  and 
less  controllable  than  the  operators  in  a  keypunch  room. 
Second,  the  verification  operation  found  in  a  keypunch  room 
is  usually  not  employed.    Third,  batch  totals  and  other  batch 
controls  often  cannot  be  used,  as  the  transactions  originate 
singly,  not  in  batches.    Fourth,  equipment  failures  will  be 
experienced.    It  is  often  when  a  terminal,  line,  or  computer 
fails,  or  during  the  recover  period,  that  errors  originate. 

"We  do,  however,  have  one  factor  that  is  strongly  in  our  favor 
in  real  time  systems.    This  is  that  in  an  appropriately  designed 
diaglogue  most  of  the  errors  made  can  be  caught  in  real  time 
as  the  operator  makes  them.    The  mistake  or  discrepancy  is  then 
rectified  on  the  spot."^'^ 

Because  of  the  differences  in  approach  between  batch  and  interactive 
systems,  the  respective  weighting  factors  are  being  handled  separately. 
(See  Form  3.2.3-1  and  3.2.3-2). 

3.2.4    Computer  Edit  Routines.    Checks  by  computer  program  of  value 
ranges,  formats,  data  consistency,  etc.,  not  only  have  the  advantage  of 
allowing  for  rapid  processing  of  the  data,  but  further  act  on  data  that 
has  already  been  stored  in  the  computer  file  and  is,  therefore,  past  most 
of  the  stages  at  which  new  errors  are  introduced.    Susan  Wooldridge  et  al . 
comments  on  the  importance  of  this  process. 
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Martin,  James,  op.cit. ,  pp.  70-71 
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"The  first  program  in  any  system  should  be  an  edit,  or  a  verifi- 
cation program.    All  data  for  a  system  should  go  into  one  simple 
edit  program,  if  at  all  possible...    The  basic  principle  of  the 
edit  program  is  that  it  detects  all  errors  that  are  logically 
possible  to  detect,  including  -  or  perhaps  especially  -  rare  and 
unlikely  ones,  for  these  are  the  errors  which,  if  allowed  to  pass 
into  the  system  cause  the  most  trouble  later  on."45 

Programmed  checks  will,  of  course,  reflect  the  characteristics  of  the 
data  in  the  file  but  may  include  format  checking,  (is  there  a  number  in 
the  numeric  field?),  range  (is  the  age  given  as  greater  than  99?), 
internal  consistency  (do  the  birthdate  and  current  age  agree?),  reason- 
ableness (would  someone  really  have  30  children?),  etc.    In  some  cases, 
an  error  situation  is  obvious  as  in  the  case  of  alphabetic  data  occurring 
in  a  numeric  field.    Where  a  determination  of  reasonableness  of  the  data 
is  required,  the  computer  program  may  not  be  able  to  make  a  final 
determination.    In  such  case  the  data  must  be  flagged  for  further  action 
by  the  data  processing  department. 

Other  important  factors  in  determining  the  likelihood  of  errors  in  the 
file  are  the  length  of  time  for  which  the  computer  editing  program  has 
been  available  as  well  as  consideration  of  the  points  in  time  when 
improvements  have  been  made  in  its  implementation.    If  only  part  of  the 
data  has  been  run  against  the  edit  program,  a  first  step  in  data  cleaning 
may  well  be  to  run  the  entire  file  through  the  editing  routine  to  assure 
that  all  obvious  errors  have  been  caught. 

Form  3.2.4-1  lists  some  of  the  principal  areas  of  error  detection  that 
can  be  performed  by  a  computer  editing  program.    The  analyst  is  encour- 
aged to  check  the  variables  in  the  current  file  against  this  list. 

3.2.5    Manual  Checks  for  Accuracy.    Valuable  as  other  checks  on  data 
validity  are,  there  is  no  substitute  for  human  judgment  in  reviewing  the 
records.    Final  determination  of  the  accuracy  of  a  record  demands  that 
the  printed  version  of  the  data  be  manually  checked  against  the  original 
document.    In  describing  the  creation  of  new  files,  Wooldridge,  Corder 
and  Johnson  state  that: 

"...after  the  data  has  been  transferred  to  magnetic  files  the 
first  time,  it  must  be  printed  out  in  full.    Every  item  must  be 
checked.    This  is  a  time-consuming  and  tedious  job;  it  is  the 
responsibility  of  the  user  department,  and  plans  should  be  made 
well  in  advance  for  laying  on  extra  staff,  authorizing  over-time, 
etc.    All  discrepancies  between  the  computer  files  and  the 
manual  ones  must  be  checked  out.    Sometimes  the  error  will  be 
in  the  old  files,  in  which  case  it  must  be  corrected  from 
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scratch...  it  is  very  dangerous  to  start  live  processing  with 
unchecked  data,  and  it  should  never  be  authorized.  Print-out 
and  checking  should  continue  for  the  first  few  cycles  of  the 
system;  undetected  program  bugs  and  input  errors  are  not  only 
likely  but  extremely  likely."^^ 

Robert  Goldstein,  in  his  Datamation  article  entitled  "The  Cost  of  Privacy 
has  suggested  that,  as  standard  procedures  are  adopted  for  reviewing  the 
contents  of  files  for  compliance  with  the  Privacy  Act,  approximately  10% 
of  the  records  will  either  be  dropped  or  rechecked  in  any  given  year. 
Under  such  a  procedure,  he  estimates  that: 


"For  a  typical  system,  about  10%  of  the  records  would  expire 
each  year,  and  90%  of  these  would  be  rechecked  rather  than 
dropped.  "47  ^« 

This  emphasizes  the  importance  of  such  manual  checks.    Richardson  and 
Cunningham  have  pointed  out  that  audit  trails  can  be  kept  with  each  data 
item  in  the  file  in  such  a  way  as  to  enhance  the  review  of  edited  data. 

"Reviewing  of  edited  data  can  benefit  considerably  from  'data- 
marking'  ,  i.e.,  storing  a  bit-string  with  every  item  to 
indicate  whether  the  item: 

(i)  has  been  queried  without  correction; 

(ii)  is  to  be  accepted  regardless; 

(iii)  has  been  corrected  manually; 

(iv)  is  a  candidate  for  automatic  correction  later; 

(v)  has  already  been  automatically  corrected. "^^ 

An  evaluation  of  the  current  file  must  include  an  analysis  of  the  type 
of  manual  checking  procedures  employed  to  date.    Form  3.2.5-1  gives  the 
reader  an  opportunity  to  score  the  variables  in  his  file  with  respect  to 
these  manual  checking  procedures. 


3.3    SUMMARY  OF  WEIGHTING  FACTORS 


Form  3.3-1  can  be  used  to  determine  final  weighting  factors  for  each  of 
the  standard  biographical  data  elements  in  the  table.    The  check  marks 
appearing  in  the  NO  columns  of  each  of  the  preceding  charts  should  be 
totalled  for  each  variable  and  the  results  entered  in  this  form.  These 
results  are  used  to  determine  candidate  fields  where  error  and  omissions 
are  likely  to  occur. 


'Ibid. ,  p.  61. 

Goldstein,  Robert  C. ,  "The  Cost  of  Privacy,"  Datamation,  October  1975, 
p.  65. 


Richardson,  M.  and  A.  D.  Cunningham,"  Vetting  of  Industrial  Survey 
Questionnaires,"  Applied  Statistics,  Vol.  2,  No.  1,  1972,  p.  54. 
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4.0    FILE  VALIDATION 


4.1  INTRODUCTION 

Agencies  currently  maintaining  personal  data  files  will  be  concerned 
with  the  development  of  procedures  for  complying  with  the  Privacy  Act 
provisions  regarding  the  relevance,  timeliness,  and  accuracy  of  data  in 
the  individual  records.    The  file  validation  techniques  thus  established 
will  also  have  implications  with  respect  to  the  evaluation  of  the  level 
of  accuracy  maintained  in  the  file  over  time.    Statistics  reflecting 
the  number  and  type  of  problems  detected  when  selected  records  are  pro- 
cessed should  give  a  clear  picture  of  improvements  in  data  handling. 
For  purposes  of  the  current  discussion,  performance  of  file  validation 
is  crucial  to  formulation  of  a  reasonably  accurate  estimate  of  the  re- 
trieval reliability  for  each  of  the  variables  maintained  in  the  records. 

4.2  VALIDATION  TECHNIQUES 

Section  3  emphasized  the  importance  of  the  source  document  with  respect 
to  the  ultimate  accuracy  level  of  the  file.    The  CUMULATIVE  WEIGHTING 
FACTOR  column  in  Table  3.3-1,  Cumulative  Weight  Factors  for  Standard 
Biographical  Data  Elements,  gives  the  total  number  of  negative  points 
accumulated  for  each  variable  during  that  analysis  procedure.    If  all 
of  those  variables  have  approximately  the  same  final  scores,  then  the 
information  gained  is  not  discriminatory,  but  rather  implies  what  is 
to  be  anticipated  in  the  area  of  accuracy  for  the  entire  file.  If, 
however,  a  few  of  the  variables  have  much  higher  scores  than  the  others, 
these  should  automatically  be  eliminated  from  consideration  as  retrieval 
keys.    These  values  are  not  as  directly  quantifiable  as  error  and 
omission  rates,  but  should  instead  serve  as  guidelines  in  the  retrieval 
key  selection  process  and,  incidentally,  as  indicators  regarding  future 
use  of  these  variables. 

Evaluation  of  error  and  omission  rates  must  be  accomplished  through  man- 
ual checks  of  the  source  documents  against  listings  of  the  records  con- 
tained in  the  computer  file.    The  first  decision  to  be  made  regarding 
this  procedure  involves  determination  of  the  number  of  documents  to  be 
checked.    A  reasonable  starting  point  would  be  to  consider  using  a  small 
test  sample,  say  100  records,  for  this  procedure,  and  then  to  select  a 
sample  size  according  to  the  procedure  given  in  Section  5.4.    It  appears 
that  a  determination  of  error/omission  rates  would  be  satisfactory  if 
the  standard  deviation  is  less  than  or  equal  to  .01.    The  procedure 
described  in  Section  5.4  is  based  on  this  degree  of  accuracy. 

Once  the  sample  size  has  been  established,  the  selection  of  the  actual 
records  can  be  made.    The  most  important  factor  here  is  that  the  records 
must  be  selected  from  different  physical  locations  within  the  data  base. 
This  is  essential  to  avoid  skewing  of  the  results  due  to  system  problems 
such  as  the  existence  of  bad  tape,  overwritten  disk  blocks,  etc.  Further, 
since  both  the  accuracy  and  the  nature  of  the  data  may  have  varied  over 
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time,  randomness  in  selection  of  records  is  essential  to  a  fair  evalua- 
tion of  the  state  of  the  data  base.    The  number  of  physical  locations 
used  in  a  given  data  base  will  depend  upon  its  size,  but  that  number 
should  not  be  less  than  four  if  a  reasonable  sampling  is  to  be  achieved. 

For  purposes  of  recording  error  and  omission  counts,  a  chart  similar  to 
that  shown  in  Table  4.2-1  should  be  created.    This  chart  may  also  serve 
as  an  audit  document  demonstrating  the  problems  detected,  resulting  error 
analysis,  problems  remaining  unresolved,  action  to  be  taken,  and,  finally, 
indication  that  the  file  has  been  corrected  and  checked.    The  analysis 
results  for  each  record  should  be  kept  on  a  separate  sheet  for  ease  of 
use  and  also  as  a  precaution  against  the  introduction  of  confusion  be- 
tween records. 


4.3    FILE  VALIDATION  EXAMPLE 

An  example  of  the  techniques  involved  in  manual  evaluation  of  the  ac- 
curacy of  the  information  contained  in  the  files  is  given  in  this  sec- 
tion. This  discussion  is  based  on  use  of  a  small  dummy  data  file  con- 
sisting of  one  hundred  records  constructed  through  random  accessing  of 
the  telephone  book  and  the  random  number  generator.  Separate  accesses 
were  made  for  each  component  of  each  name  and  for  each  number  group. 
Thus  the  entries  are  entirely  fictitious  . 

The  ten  records  (in  this  case  ten  percent  of  the  file)  selected  for 
validation  are  shown,  in  the  most  accurate  source  format  available,  in 
Table  4.3-1.    This  data  is  assumed  to  exist  on  the  application  forms 
filled  out  by  each  of  the  individuals.    It  is  also  assumed  for  purposes 
of  illustration  that  the  printouts  shown  in  Table  4.3-2  were  obtained 
for  each  of  these  records  from  the  computer  file. 

Table  4.3-3  contains  the  results  of  the  manual  check  performed  on  records 
number  20  and  30.    In  this  case,  data  has  been  stored  in  the  file  for  a 
set  of  twins,  HECTOR  and  HARRY  GENERAUX.    As  is  clear  from  the  source 
documents,  the  twins  lived  at  the  same  address  at  the  time  the  applica- 
tions were  filled  out.    In  the  intervening  period,  HECTOR  moved  to  Laurel 
Street,  and  a  system  update  was  initiated  by  the  operator  when  he  re- 
ported his  new  address.    The  update  to  the  record  was  made  on  the  basis 
of  SURNAME,  and  BIRTHDATE,  and  thus  it  was  easy  for  the  mistake  to  occur. 

In  record  number  70  an  obvious  case  of  number  transposition  has  occurred 
in  the  address  field,  in  this  case  affecting  both  street  and  house 
number.    Record  80  has  a  mistake  in  the  spelling  of  the  SURNAME  where 
an  'I*  has  incorrectly  been  input  as  'E'.    And  in  record  100,  a  mistake 
has  occurred  in  entering  the  birthdate.    These  errors  are  summarized  in 
Table  4.3-4. 

Use  of  these  rates  is  discussed  in  Section  5. 
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5.0    COMPUTER  SIMULATION  OF  DATA  BASE  RETRIEVAL 


5.1    PRECISION  AND  RECALL 

The  preceding  sections  have  introduced  methodologies  for  determining  the 
reliability  of  currently  existing  data  bases.    It  was  seen  that  a  large 
part  of  the  privacy  problem  reduces  to  the  technological  capabilities  of 
information  storage  and  retrieval  systems.    This  section  therefore  deals 
with  concepts  related  to  these  questions,  such  as  effectiveness  of  re- 
trieval techniques,  precision,  and  recall.    Finally,  the  quantification 
of  these  concepts  provides  tools  for  information  systems  managers  to  use 
in  evaluation  of  the  effectiveness  of  their  systems.    Computer  simula- 
tions are  a  powerful  aid  in  performance  of  such  evaluations.  Accordingly, 
a  mathematical  model  and  its  associated  computer  simulation  are  presented 
in  this  section. 

As  noted  in  Section  2.4, the  effectiveness  of  information  storage  and  re- 
trieval systems  has  classically  been  measured  by  two  parameters:  recall 
and  precision.    Recall  is  defined  as  the  percentage  of  relevant  records 
which  are  retrieved  in  response  to  a  particular  query,  while  precision 
is  the  percentage  of  the  retrieved  records  which  are  relevant.    An  ideal 
system  returns  100%  of  the  relevant  records  in  the  data  base  in  response 
to  each  query,  and, in  that  subset  of  the  data  base  retrieved  by  the  system, 
100%  of  the  records  are  relevant.    Such  high  levels  of  recall  and  precision 
can  be  simultaneously  achieved  under  the  following  conditions: 

1.  Queries  refer  only  to  key  fields  where  the  data  is  perfectly 
recorded  (no  errors  or  omissions). 

2.  All  of  the  records  in  the  data  base  are  examined  to  determine 
if  they  meet  the  query  criteria. 

Many  storage  and  retrieval  systems  are  operated  as  if  these  conditions  were 
true  when,  in  fact,  they  are  known  to  be  false.    Condition  1,  perfectly 
valid  data,  can  be  approached,  but  never  really  reached;  condition  2, 
examination  of  all  records  in  the  data  base,  can  be  simulated  by  the  use 
of  indexed  file  structures  and  multi-level  searches.    Indeed,  when 
queries  consist  of  only  terms  upon  which  indexes  have  been  built,  condi- 
tion 2  holds  for  real  data  bases. 

It  should  be  pointed  out,  however,  that  the  recall  and  precision  figures 
discussed  in  the  preceding  paragraph  refer  only  to  the  relationship  be- 
tween the  query  as  stated  to  the  system  and  the  retrieved  records  which 
resulted  from  the  query.    Normally  this  process  is  simply  a  surrogate 
for  the  more  complicated  process  of  locating  the  record  or  records  in  a 
data  base  which  match  some  aspect  of  the  real  world.    Often  the  criteria 
by  which  the  user  wishes  to  select  a  record  are  not  those  criteria  which 
were  embodied  in  the  indexing  system  of  the  data  base,  thus  causing  the 
user's  query  to  be  translated  into  one  or  more  system-oriented  queries 
before  the  search  is  carried  out.    Under  these  conditions,  the  most 
interesting  queries  are  likely  to  be  those  which  the  system  cannot  answer. 
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and  which  are  thus  not  translatable  into  equivalent  system  oriented  query 
foraiulations.    Consequently,  the  system-oriented  translations  of  such 
queries,  even  if  answered  with  100%  recall  and  precision,  do  not  provide 
high  levels  of  recall  and  precision  with  respect  to  the  real  query 
that  specified  by  the  user. 

Errors  in  the  data  base  become  most  important  when  queries  are  made  very 
specific  and  when  total  recall  (with  respect  to  the  real  world,  not  just 
to  the  query)  is  required.    When,  for  example,  there  is  a  statutory  re- 
quirement to  identify  the  records  in  a  data  base  which  contains  personal 
data  in  order  to  provide  benefits  to  those  persons  who  meet  the  necessary 
qualifications,  there  is  at  least  a  social  cost  associated  with  neglect- 
ing to  properly  identify  those  individuals  who  should  be  provided  with 
the  benefits,  and  there  is  an  economic  cost  which  results  from  improperly 
providing  benefits  to  ineligible  recipients.    There  is  also  a  statutory 
requirement  that  agencies  which  maintain  personal  data  bases  provide, 
upon  demand,  the  records  which  pertain  to  an  individual  and  allow  the 
subject  of  such  records  to  challenge  errors  of  fact,  etc.    The  problem 
here  is  one  of  providing  the  inquirer  with  his,  and  only  his,  record, 
without  requiring  him  to  provide  sufficient  information  to  create  a 
record  on  him  if  it  does  not  already  exist. 

Errors  in  the  data  base  may  be  of  two  types:    destructive,  where  the  er- 
roneous data  in  a  particular  field  errs  in  such  a  way  that  it  will  not 
satisfy  any  real  query  against  the  data  base,  and  transformational,  where 
the  content  of  a  data  field  is  wrong,  but  satisfiable.    An  example  of  the 
first  type  of  error  would  be  something  like  AGE=2B  (alpha  data  in  a  numeric 
field),  which  will  never  match  a  numeric  query  statement.    An  example  of  a 
transformational  error  is  NAME=SMITH  (when  the  name  is  really  SMYTHE). 
Destructive  errors  cause  a  record  to  become  inaccessible  via  the  particular 
key  field  in  which  the  key  value  has  been  destroyed,  while  transformational 
errors  will  cause  the  record  to  be  returned  in  response  to  queries  for 
which  it  is  irrelevant,  and  to  be  missed  when  it  should  be  found.  The 
effect  of  error  rates  in  the  various  data  fields  employed  in  a  query  can 
be  quantified  as  follows: 

1.  Given  a  query  Q  consisting  of  the  conjunction  of  n  keys 

K^,  ...  K^,  where  the  K^.  each  represent  a  parti- 
cular value  of  a  particular  key  field  in  the  data  base 
records,  and 

2.  If  each  of  the  fields  in  the  query  Q  is  associated  with 
confidence  factors  (equal  to  1  minus  the  sum  of  error 
and  omission  rates)  c,  ,  c^,  ...  c  , 
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then  the  recall  ratio  associated  with  the  response  to  query  Q  is  the  pro- 
duct of  the  confidence  factors  associated  with  each  of  the  key  fields:* 


n 

3.    G  =   TT  c, 


i=l 


It  follows  that  queries  which  reference  more  key  fields  in  an  attempt  to 
achieve  more  precision  tend  to  be  subject  to  smaller  recall  ratios  since 
the  c^.  are  always  less  than  or  equal  to  1. 

5.2    QUERY  FORMULATION  FOR  BEST  RESULTS 

Queries  should  be  formulated  using  the  most  restrictive  key  fields  in 
order  to  provide  the  most  specific  answers.    Additional  precision  can  be 
achieved  by  utilizing  the  conjunction  of  more  fields  in  each  query,  al- 
though this  may  result  in  a  decline  in  the  recall  ratio,  thus  causing 
records  to  be  missed  which  are  actually  contained  in  the  data  base.  Equal- 
ly, those  fields  with  the  lowest  error  rates  (i.e.,  with  the  highest  con- 
fidence factors)  should  be  preferred  over  those  known  to  be  badly  in  error. 
Qualitatively,  the  following  order  provides  the  greatest  discrimination 
among  records: 

1.  Name  (Surname  and  Given  Name),  although  the  discriminatory 
power  of  this  field  varies  depending  on  the  nature  of  the 
surname.    Matching  a  name  like  TREFFTZS  provides  much  more 
information  than  matching  a  name  like  SMITH,  for  example. 

2.  Surname. 

3.  Residence  address  (Street  Number,  Street  Name,  City,  and 
State),  although  this  information  tends  to  decay  rapidly 
with  time. 


*Recall  ratio  is  defined  as  the  proportion  of  relevant  records  retrieved 
in  response  to  a  query,  or  equivalently ,  as  the  probability  of  a  relevant 
record  being  retrieved.    For  the  purpose  of  this  discussion  (as  pointed 
out  above),  a  relevant  record  is  one  corresponding  to  an  individual  who 
has  exactly  those  attributes  defined  by  the  query.    However,  the  actual 
recorded  key  values  may  either  be  in  error  or  omitted.    Thus,  the  pro- 
bability of  the  key  value  in  the  ith  key  field  being  correct,  and  hence 
retrievable  by  the  query,  is  c-j .    If  the  retrieval  mechanism  is  such 
that  a  record  is  retrieved  only  if  it  matches  on  each  of  n  key  values 
in  a  query,  then  it  follows  that  the  probability  of  a  relevant  record 
being  retrieved  is  the  product  of  all  the  n  probabilities;  i.e.. 


and  so  this  is  the  recall  ratio. 
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4. 


Date  of  Birth  (Year,  Month,  and  Day). 


5.  Place  of  Birth  (City,  State),  although  this  also  varies: 
Sparta,  Illinois  as  a  birthplace  provides  more  identify- 
ing information  than  does  New  York,  New  York. 

6.  Sex.    This  is  an  example  of  a  key  field  which  provides 
very  little  information       it  may  partition  the  file  into 
two  equal  halves  at  best.    In  more  extreme  cases  where 
the  contents  of  the  file  are  heavily  skewed  (all  males 

or  all  females)  this  field  provides,  in  general,  even 
less  information,  only  being  useful  in  the  rarer  cases. 

Subjective  estimates  on  the  discrimination  of  various  keys  are  provided 
in  Table  5.3-1  while  statistical  data  on  surname  records  are  given  in 
Table  5.3-2. 

5.3  SOFT  MATCH  TECHNIQUES 

"Soft-Match"  techniques  may  be  utilized  to  overcome  the  decline  in  recall 
which  is  normally  associated  with  improvements  in  precision.    The  increas- 
ed error  rate  which  is  associated  with  increases  in  the  number  of  variables 
entering  into  the  query  can  be  reduced  by  examining  the  records  retrieved 
by  the  sub-queries.    For  example,  a  query  containing  three  key  fields  would 
ideally  return  a  record  (or  set  of  records)  in  which  the  three  required 
values  are  found.    Errors  in  the  recordirjig  of  either  the  original  data 
or  the  query  may,  however,  have  produced  only  a  partial  match,  i.e.,  for 
either  one  or  two  keys.    The  problem  is  to  devise  a  method  for  accounting 
for  these  partial  matches.    Such  methods  are  called  "Soft-Match"  techniques. 

The  result  of  a  Soft-Match  Technique  is  not  only  a  set  of  retrieval  records, 
but  also  a  numerical  weight  assigned  to  each  record  to  indicate  its  degree 
of  responsiveness  to  the  query.    In  Appendix  III  a  specific  weighting 
scheme  for  a  Soft-Match  Technique  is  presented. 

5.4  EXAMPLE  OF  USE  OF  FILE  VALIDATION  SUMMARY  DATA 

In  Section  4.3,  an  example  of  the  use  of  file  validation  techniques  is 
given.    Tables  4.3-4  detailed  the  summary  of  error  and  omissions  within 
the  sample  data  file.    The  final  tally  of  errors  shown  in  the  table  for 
each  of  the  variables  is: 


Variable 


Error  Count  Per  10 


SURNAME 


1 


FIRST  NAME 


0 


STREET  NUMBER 


3 


STREET  NAME 


3 


DATE  OF  BIRTH 


1 
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Table  5.3-1    Retrieval  Ratios  (Estimates  of  Proportion  of  File  Retrieved)* 


Field  Number  Contents  Retrieval  Ratio 


1 

Day  and  Month  of  Birth 

1/365 

2 

Month  of  Birth 

1/12 

3 

Year  of  Birth 

1/50 

4 

State 

1/50 

5 

Sex 

1/2 

6 

Street  Number 

1/10,000 

7 

Street  Name 

1/5,000 

8 

Education 

1/15 

9 

County 

1/3,000 

10 

Date  of  Birth  (YMD) 

1/18,000 

*The  values  used  are  based  on  subjective  estimates  taking  into  consider- 
ation the  meaning  of  the  keys.    These  values  are  used  in  the  model  de- 
scribed in  Section  5.5. 
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Table    5.3-2    Statistical  Classification  of  Surname  Records  as  compiled 
from  Social  Security  Administration  Accounts. 


DECILE*       Number  of  Records  Retrieved'-^*    %  of  File  Retrieved** 


n 
1 

io.  Jo 

. ooouoyo/A 

o 

/I  no   1  "7 
4Uo. 1/ 

.0002459/6 

3 

1,599.28 

.0009635% 

4 

4,515.456 

.00272037% 

5 

8,000.00 

.00481966% 

6 

10,999.94 

.006627% 

7 

20,997.32 

.01265% 

8 

80,000.62 

.048197% 

9 

130,000.80 

.07832% 

10 

561,773.3 

. 3384% 

*Decile  1  corresponds  to  the  least  common  names.  Decile  10,  the 
most  common  (e.g.,  Smith). 

*Based  on  1  value  per  surname  key.  Each  decile  contains  10% 
of  the  population. 


"Report  of  Distributions  of  Surnames  in  the  Social  Account  Number 
File,"  Department  of  Health,  Education  and  Welfare,  June  1,  1964. 


64 


These  error  rates  imply  that  the  file  as  a  whole  is  heavily  riddled  with 
serious  errors:    the  single  error  in  the  surname  sample  implies  that  10 
of  the  records  in  the  file  are  coded  with  an  erroneous  SURNAME  --  this 
is  10%  of  the  file.    Three  detected  errors  in  the  STREET  NUMBER  again 
imply  that  30%  of  the  street  numbers  in  the  file  as  a  whole  are  in  error, 
etc.    Based  on  this  information,  a  typical  query  against  this  file,  using 
the  keys  of  SURNAME,  GIVEN  NAME,  STREET  NUMBER,  STREET  NAME,  and  DATE  OF 
BIRTH,  would  have  a  combined  confidence/recall  factors  of  only  39.69%  -- 
less  than  half  of  the  truly  relevant  records  would  ever  be  located  in 
response  to  a  query. 


Normally,  of  course,  statistical  methods  would  not  be  used  on  a  file  as 
small  as  the  dummy  data  base  discussed  here,  but  would  be  used  for  a  file 
containing  thousands  of  records.    The  procedure  is  as  follows: 

Suppose  we  want  to  determine  the  error  rate  (or  the  omission  rate  --  the 
theory  is  the  same)  so  that  the  true  value  differs  from  the  measured 
value  by  less  than  0.01  with  a  high  probability  (say  66%).    This  means 
we  want  to  choose  a  sample  size  so  that  the  standard  deviation  in  the 
determination  of  the  error  rate  is  less  than  0.01.    The  standard  deviation 
is  given  by  the  formula: 50 


error  rate 


error  rate  x  (1  -  error  rate) 
Sample  Size 


Therefore,  if  we  wish  the  standard  deviation  to  be  less  than  0.01,  we 
must  have 


Sample  size  >  10,000  x  error  rate  x  (1  -  error  rate). 


Since  the  product  of  the  error  rate  and  its  complement  is  bounded  by  0.25, 
a  sample  size  of  2,500  will  assure  a  standard  deviation  of  less  than 
0.01.    But  a  smaller  sample  size  may  be  used,  depending  on  the  magnitude 
of  the  error  rate.    To  determine  this,  select  a  small  test  sample,  say 
of  100  records.    Estimate  the  error  rate  for  this  test  sample.    Then  a 
safe  determination  of  an  adequate  sample  size  is  given  by 


Sample  size  =  10,000  x  estimate  of  error  rate. 


For  example,  if  the  estimate  is  0.1,  then  the  sample  size  should  be  1000. 

Having  determined  the  error  and  omission  rates,  the  confidence  factors 
(see  Section  5.1)  can  be  calculated.    The  recall  ratio  for  any  query  in- 
volving the  key  fields  for  which  these  confidence  factors  have  been 
developed  is  simply  the  product  of  the  confidence  factors. 


Hoel,  P.  G. ,  Introduction  to  Mathematical  Statistics,  John  Wiley  and 
Sons,  New  York,  1954,  pp.  87-88,  also  pp.  107-108.    This  is  the  usual 
formula  for  the  standard  deviation  of  a  mean  value. 
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Assuming  that  the  error  and  omission  rates  for  the  variables  SURNAME, 
STREET  NUMBER,  STREET  NAME,  and  DATE  OF  BIRTH  were  0.01,  0.03  ,  0.03  ,  0.01, 
respectively,  then  the  resulting  confidence  factors  would  be: 

SURNAME  0.99 

STREET  NUMBER  0,97 

STREET  NAME  0.97 

DATE  OF  BIRTH  0.99 

and  a  query  involving  all  four  of  these  fields  would  tend  to  retrieve 
0.99  X  0.97  X  0.97  x  0.99  =  0.922  or  92.2%  of  the  relevant  records. 

Use  of  these  data  will  be  shown  below. 

5.5    PRECISION  AND  PROBABLE  MATCH 

Given  a  certain  retrieved  record,  the  question  arises:    Is  it  in  fact 
the  correct  record  for  the  requester?    The  answer  to  this  must  be  ex- 
pressed as  a  probability.    The  theoretical  formulation  of  this  probabil- 
ity problem  is  presented  in  Appendix  II. 

A  computer  program  was  written  to  calculate  probability  values  resulting 
from  the  use  of  various  combinations  of  keys.    Recall  that  the  basic  pro- 
blem is  to  evaluate  the  probability  that  a  record  will  match  a  requester 
given  the  presence  in  the  record  of  a  number  of  key  values  K, ,  K 
which  match  the  request.    The  probability  values  are  developed  in  terfns 
of  the  following  auxiliary  quantities. 

(1)  p  =  the  a  priori  probability  that  a  randomly  selected  record 
corresponds  to  that  record  desired  by  a  requester. 

(Notation  in  Appendix  II,  Equation  ( 16),  p  =  P(M^. )) 

The  range  of  p  should  be  from  1/T  (where  T  is  the  largest  population  from 
which  the  data  base  is  selected)  to  1/N  (where  N  is  the  number  of  records 
in  the  data  base). 

For  each  key  j  on  which  the  search  is  conducted,  let 

(2)  a.  =  error  rate  in  key  j  (a  proportion). 

(3)  b.  =  relative  frequency  of  omission  of  data  in  key  j. 
From  (2)  and  (3),  we  calculate 

(4)  cj  =  1  -  aj  -  bj. 
Thus  c.  is  the  probability 
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P(Kj|M.) 

given  in  Appendix  II,  Equation  (8b), 


(5)  Let  n.  be  the  number  of  records  with  specific  matching  data  in 
key  j,  i.e.,  the  number  of  records  retrieved  matching  the  re- 
quested value  of  key  j.    Let  N  be  the  number  of  records  in  the 
data  base,  then  define 

r.  -  n./li 

Thus  r.  is  the  probability  P(K.)  given  in  Appendix  II,  p.  176. 

(6)  For  each  key  value  j  we  define 

(7)  We  then  have  from  Appendix  II,  Equation  (18),  the  probability 
P(M.  |K^.)  given  by 

(8)  An  auxiliary  quantity  F  (Appendix  II,  Equation  (23))  is  then 
calculated: 

where  n  is  the  number  of  key  values  which  match  the  request.    The  quan- 
tity F  is  used  as  shown  below. 

(9)  Finally,  the  required  probability  (i.e.,  the  probability  that 
a  record  is  that  desired  by  a  requester  conditional  on  a  match 
on  n  key  values  in  the  query)  is  given  by 

V  = 


1  +  EF 


where  E  is  a  quantization  of  a  certain  independence  relation  between  the 
keys,    (see  Appendix  II,  pp.  179-180).    The  smaller  the  value  of  E,  the 
greater  the  ability  of  the  keys  to  discriminate.    Independent  keys  mean 
that  E  =  L    If  E  <  1,  then  V  >  1/(1  +  F). 

The  probability  value  V  corresponds  to  the  precision  of  a  retrieval  of 
one  record. 

Results  of  the  computer  simulation  are  shown  in  Appendix  I.    The  tables 
should  be  read  as  follows: 
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(1)  Each  table  is  identified  by  the  data  fields  used  in  the 
simulated  query.    All  simulated  queries  utilized  the  sur- 
name field. 

(2)  The  surname  decile  identifies  the  discriminatory  power  of 
the  particular  surname  used  in  the  query:    Surnames  in 
the  first  decile  are  the  most  specific  (TREFFTZS,  for 
example)  while  those  in  the  tenth  decile  are  the  more 
common  surnames j  such  as  JONES  and  SMITH. 

(3)  The  column  headings  reflect  the  three  sample  file  sizes 
which  were  simulated:    1  million,  4  million,  and  220 
million  records. 

(4)  The  left  column  of  the  table  reflects  the  recall  factor* 
obtained  by  multiplying  together  the  simulated  error  rate 
for  each  of  the  fields  utilized  in  the  query.    These  rates 
were  varied  in  parallel  from  0  (no  errors)  to  6%  errors 

in  increments  of  0.5%.    All  fields  were  assumed  to  be 
subject  to  the  same  error. 

(5)  The  body  of  each  table  tabulates  the  precision  of  each 
retrieved  result  with  respect  to  the  query.    Values  of 
1.0  imply  that  unique  records  were  properly  identified. 
Values  less  than  1.0  imply  that  the  probability  of 
unique  identification  is  less  than  100%. 

The  model  reflects  the  precision  and  recall  resulting  from  the  attempt 
to  locate  the  record  pertaining  to  a  specific  individual  in  a  data  base 
of  the  specified  size,  subject  to  given  error  rates,  for  several  types 
of  hypothetical  queries.    Three  file  sizes  are  postulated:    a  file  of 
220  million  records,  one  of  4  million  records,  and  a  smaller  file  con- 
taining only  1  million  records.    All  the  queries  are  assumed  to  consist 
of  a  surname  and  one  or  more  other  key  values.    The  discriminatory  power 
of  a  query  varies  with  the  discriminatory  power  of  the  key  and  key  values 
used  and  the  confidence  factor  in  the  data  base.    The  precision  of  a 
retrieval  made  against  the  three  file  sizes  given  the  retrieval  confi- 
dence factor  of  92.2%  in  the  example  of  Section  5.4,  and  utilizing  the 
four  key  fields  of  surname,  street  number,  street  name,  and  date  of  birth 
can  be  read  from  the  fifth    line  (recal 1  0.922)  of  Tables  5a  -  5j  in 
Appendix  L    The  tables  present  10  different  values  for  the  retrieval 
precision  of  such  a  query,  depending  on  the  value  of  the  surname:  names 
such  as  Jones  have  far  less  discriminatory  power  than  names  such  as 
Trefftzs  or  Krcmar.    It  can  be  seen  that  the  precision  of  a  query  im- 
proves as  the  file  size  decreases.    Files  smaller  than  1  million  records 
will  provide  proportionately  better  results. 


The  definition  of  recall  is  given  in  Section  5.2. 
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6.0    CONCLUSIONS  AND  RECOMMENDATIONS 


6.1  CONCLUSIONS 

The  preceding  sections  have  addressed  the  problem  of  accessing  individual 
records  from  personal  data  files  using  non-unique  identifiers  where 
accessing  includes  updating,  purging,  and  other  bookkeeping  tasks.  The 
value  of  the  legislation  requiring  Federal  agencies  to  make  sych  records 
available  on  request  to  individuals  whose  dossiers  exist  in  their  files 
is  obvious,  but  compliance  with  this  legislation  is  a  complex  matter  for 
which  there  can  be  no  single  generalized  approach.    Each  agency  must 
determine  independently  the  most  appropriate  solution  given  its  own 
operational  environment. 

Exphasis  in  this  document  has  been  on  one  aspect  of  the  required  imple- 
mentation:   the  retrieval  of  the  individual's  record  through  use  of 
identifiers  that  he  can  readily  recall  and  that  do  not  include  his  Social 
Security  account  number.    The  analysis  has  included  examination  of  the 
characteristics  of  standard  identifiers  with  emphasis  on  the  varying 
selectivity  of  these  variables.    The  high  discriminatory  value  of  the 
NAME  variable  resulted  in  inclusion  of  descriptions  and  evaluations  of 
several  of  the  currently  available  name  lookup  algorithms.  Techniques 
applicable  to  this  type  of  retrieval  problem  were  discussed  along  with 
indexing  schemes  ranging  from  the  most  straightforward  to  complex  systems 
of  encoded  pointers. 

The  selected  examples  for  generating  the  precision  tables  from  the  mathe- 
matical model  (see  App.  I)  clearly  show  that  retrieval  through  use  of 
non-unique  identifiers  is,  indeed,  possible.    It  must  be  understood, 
however,  that  the  search  keys  used  in  these  examples  are  illustrative 
only.    In  practical  cases  it  may  be  necessary  to  employ  search  keys  with 
more  discriminating  power.    Furthermore,  such  retrieval  has  been  carried 
out  in  a  variety  of  ways  and  has  proved  to  be  practical.    For  example, 
the  3-level  index  scheme  developed  by  the  Immigration  and  Naturalization 
Service  presently  retrieves  5,000,000  records  (anticipated  growth  up  to 
10,000,000  records)  with  a  response  time  per  record  measured  in  seconds. 

The  discussion  of  available  techniques  also  made  it  clear  that  the  degree 
of  success  resulting  from  use  of  such  techniques  depends  directly  on  the 
quality  of  the  dato  items  being  used  as  search  keys.     (The  assumption 
is  made  that  the  query  data  itself  if  correct  and  that  it  has  been 
entered  properly.)    The  reliability  of  retrieval  through  use  of  non-unique 
identifiers  was  further  demonstrated  through  use  of  a  probability  model 
programmed  to  generate  the  precision  (i.e.,  the  probability  that  a  record 
if  retrieved  as  matching  the  query,  as  formulated,  .on  a  variety  of  keys, 
will  be  in  fact  the  correct  record),  given  a  wide  range  of  recall  factors 
(i.e.,  the  probability  that  the  record,  if  in  the  file,  will  be  retrieved 
based  on  a  range  of  confidence  factors  simulating  a  reasonable  level  of 
error  and  omission  rates).    Results  from  this  modeling  procedure  again 
indicate  that  retrieval  mechanisms  based  on  high  discrimination  keys 
are  practical . 
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It  is  also  clear,  however,  that  these  techniques  must  be  implemented  with 
great  care.    The  use  of  non-unique  identifiers  results  in  very  high  pre- 
cision but  rather  low  recall.    This  effect  is  obvious  from  considera- 
tion of  the  fact  that  the  more  identifiers  used,  the  more  complete  the 
identification  of  the  individual  becomes.    At  the  same  time,  the  intro- 
duction of  more  keys  increases  the  potential  for  problems  in  matching 
exactly  the  information  that  is  contained  in  the  file.    For  this  reason, 
a  Soft-Match  Technique,  such  as  that  described  in  Section  5,  may  be 
necessary  as  back  up  to  the  retrieval  procedure. 

6.2  RECOMMENDATIONS 

The  Office  of  Management  and  Budget  has  interpreted  the  language  of  the 
Privacy  Act  to  mean  that  the  mere  existence  of  personal  data  items  in  a 
file  is  not  sufficient  to  bring  it  under  these  legislative  constraints. 
In  accordance  with  the  definition  of  "system  of  records"  (Privacy  Act, 
Sec.  552a  (a)  (5))  these  items  become  relevant  only  when  they  are  used 
as  retrieval  keys: 

"This  language  further  suggests  that  the  Congress  did  not 
intend  to  require  that  an  individual  be  given  access  to 
information  which  the  agency  does  not  retrieve  by  reference 
to  his  or  her  name  or  some  other  identifying  particular." 

A  first  step  in  compliance  will  then  be  to  determine  the  exact  keys  cur- 
rently in  use  and  from  that  list  to  determine  whether  or  not  the  legis- 
lation applies. 

The  current  considerations  apply  only  to  the  specific  problem  of  the 
retrieval  of  an  individual's  record.    But  even  this  single  aspect  of 
the  legislation  will  require  a  review  of  existing  systems  to  make  the 
following  determinations: 

1.  Are  the  personal  identifiers  contained  in  the  file  necessary? 

2.  What  non-unique  identifiers  are  currently  available  as 
retrieval  keys? 

3.  Are  these  currently  available  retrieval  keys  adequate  for 
accessing  the  required  records  in  a  cost  effective  way? 

4.  If  not,  how  can  the  existing  capabilities  be  expanded  or 
modified  to  make  the  required  records  accessible? 

Each  of  these  considerations  is  discussed  below. 

51  Office  of  Management  and  Budget  Circular  A-108  and  accompanying 
"Guidelines  for  Implementing  Section  3  of  the  Privacy  Act  of  1974," 
Federal  Register,  9  July  1975,  Vol.  40,  No.  32,  pp.  28948-28978. 
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6.2.1  Evaluation  of  Current  File  Access  Capability.    If  it  does  apply, 
the  agency  should  make  a  determination  of  acceptability  of  the  current 
retrieval  capability  in  the  context  of  the  legislation  and  the  0MB  Guide- 
lines.   Using  the  search  keys  that  are  both  available  and  relevant, 
weighting  factors  should  be  formed  as  outlined  in  Section  3.    This  pro- 
cedure should  be  followed  by  an  analysis  of  the  error  and  omission  rates 
for  these  variables  as  suggested  in  Section  4.    From  these  rates,  confi- 
dence factors  can  be  formed,  using  the  methodology  described  in  Section  5, 
and  a  determination  of  precision  and  recall  made  for  some  combinations 

of  these  variables  using  the  computer-generated  tables  shown  in  Appendix 
I.    If  the  variables  that  are  already  available  as  retrieval  keys  are 
not  adequate  to  the  task,  then  current  system  capabilities  will  require 
expansion.    Some  discussion  of  available  techniques  is  given  in  Section 
2  for  consideration  under  these  circumstances.    It  should  be  noted,  how- 
ever, that  the  Office  of  Management  and  Budget  has  stressed  use  of  exist- 
ing capabilities  whenever  possible. 

"...the  development  of  new  retrieval  and  indexing  capa- 
bilities is  not  encouraged,  rather  agencies  should  ex- 
ploit existing  capabilities  to  serve  individual  needs. "52 

6.2.2  Evaluation  of  the  Data  Base.    The  most  difficult  aspect  of  the 
analysis  procedure  is  that  involving  evaluation  of  the  data  base  itself. 

An  orderly  sequence  of  steps  should  be  taken  to  evaluate  any  data  base 
in  order  to  bring  it  into  compliance  with  the  requirements  of  the  Privacy 
Act.    A  similar  sequence  should  be  followed  when  establishing  any  new 
data  bases  containing  personal  data.    For  existing  data  bases,  the  re- 
quired actions  fall  into  the  categories  of: 

1.  Analysis  of  present  data  validity, 

2.  Correction  of  errors, 

3.  Retirement  of  obsolete  data,  and 

4.  Establishment  of  a  review  cycle  to  keep  data  current. 

For  new  data  bases,  the  first  of  these  steps  is  replaced  by  an  analysis 
of  data  requirements  and  initial  data  collection  steps. 

While  it  is  unrealistic  to  assume  that  any  data  base  is  error-free  in 
any  data  field,  all  fields  which  comprise  the  data  base  should 
contain  data  which  are  as  accurate  as  possible.    Error  rates  which  ex- 
ceed four  or  five  percent  tend  to  throw  the  utility  of  the  data  base  into 
question,  and  fields  with  such  error  rates  should  be  either  corrected 
(at  some  cost)  or  deleted  from  the  file.    The  basic  test  of  any  personal 
data  base  is  whether  the  people  represented  by  the  records  in  the  data 
base  can,  in  fact,  be  found  and  shown  to  be  the  individuals  to  whom  the 
records  are  assumed  to  refer.    As  time  passes,  addresses  change, 
names  change,  ages  change,  occupations  change,  and  people  move. 


Loc.Cit. 
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Files  which  have  not  been  subjected  to  ongoing  verification  and  update 
activities,  particularly  those  in  which  the  subjects  are  not  actively 
involved  in  the  maintenance  process,  may  be  found  to  contain  a  large 
cast  of  fictional  characters. 

The  requirement  for  the  maintenance  of  such  files  should  be  reviewed 
at  the  highest  levels--certainly  such  information  should  not  be  made 
the  basis  of  substantive  policy  decisions,  and  the  very  need  for  the 
file  in  question  should  be  carefully  considered.    Files  which  are  really 
necessary,  however,  should  be  evaluated  periodically  to  determine  their 
accuracy  and  to  correct  erroneous  data. 

The  utility  of  such  files--and  compliance  with  privacy  legislation-- 
requires  that  a  high  level  of  accuracy  be  maintained,  and  that  suitable 
resources  be  expended  to  insure  such  a  level  of  accuracy. 
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APPENDIX  I 
PRECISION  TABLES 


Table  IDs  Variables  Selected   Pages 

la  -  Ij     Surname  and  Sex^   74  -  83 

2a  -  2j     Surname  and  Street  Number   84  -  93 

3a  -  3j     Surname  and  Birthdate  (month/day)    94-103 

4a  -  4j     Surname  and  Birthdate  (year/month/day)   104  -  113 

5a  -  5j  Surname ,  Birthdate  (year/month/day).  Street  Name 

and  Street  Number    114-123 

6a  -  6j     Surname  and  State   124-133 

7a  -  7j     Surname,  Sex,  and  Birthdate  (month/day)    134-143 

8a  -  8j     Surname,  Sex,  and  Birthdate  (year/month/day)   144  -  153 

9a  -  9j  ^  Surname,  Street  Name,  Street  Number,  and  State .  .  .  .  154  -  163 

10a  -  lOj  Surname,  Sex,  Street  Name,  Street  Number,  and 

Birthdate"Tyear/month/day)   164-173 
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APPENDIX  II 

A  PROBABILISTIC  FORMULATION  OF  THE  NON-UNIQUE  ACCESS  PROBLEM 


In  this  appendix,  we  describe  the  theoretical  basis  of  the  probability 
model  discussed  in  Section  5.    The  problem  is  to  calculate  the  prob- 
ability that  a  given  retrieved  record  which  has  been  accessed  by  a 
combination  of  retrieval  keys  such  as  name,  date  of  birth,  etc.,  will 
indeed  correspond  to  an  individual  whose  record  is  requested.    It  is  to 
be  noted  that  this  theoretical  development  is  completely  independent 
of  the  retrieval  algorithm  itself.    It  supposes  that  a  certain  record 
has  been  retrieved  and  that  it  has  certain  characteristics  which  match 
characteristics  known  by  the  inquirer.    The  question  is  then:    what  is 
the  probability  that  it  is  in  fact  the  record  desired  by  the  inquirer? 

Note  that  there  is  some  conceptual  difficulty  in  even  defining  precisely 
when  a  record  is,  in  fact,  the  record  of  a  particular  person.    This  is 
because  the  record  may  not  uniquely  characterize  a  person  or  may  contain 
erroneous  information.    There  are  also  attributive  values  which  carry 
more  weight  than  others  in  identifying  a  record-person  correspondence 
(e.g.,  name).    For  purposes  of  the  discussion,  however,  we  assume  that 
it  is  meaningful  to  say  that  a  record  matches  or  corresponds  to  an 
individual,  i.e.,  is^  the  record  of  the  individual.    This  proposition 
or  event  we  designate  as  'M. ';  that  is, 


The  second  event  to  be  specified  in  the  probabilistic  formulation  is  that 
a  record  contains  a  particular  attribute  value  for  the  jth  retrieval  key: 


It  is  useful  to  define  a  third  "reference"  event  that  defines  the  universe 
of  discourse  or  context  of  the  probability  formulation.    We  call  this  A. 

In  what  follows  we  can  consider  A  to  be  the  selection  of  a  given  record 
from  a  data  bank  where  the  mode  of  selection  is  not  specified. 

The  basic  probability  to  be  calculated  is  then 


M.  =  a  given  record  matches  the  individual  i 


(1) 


(e.g.,  who  has  attributes  name  =  Xp  address 
=  x^,         attribute^  =  x^) 


Kj  =  a  given  record  has  a  particular  attribute 
value  for  key  j  (e.g.,  name  =  x, ) 


(2) 


P(M. |Ki  &  K2  &  . . .  &       &  A) 


(3) 
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where  '&'  indicates  logical  conjunction.    Thus  (3)  is  the  probability 
that  a  given  record  matches  the  individual  i  conditional  on  the  fact 
that  the  record  contains  certain  attribute  values  on  the  set  of  keys 
j  =  1,  n.    The  basic  problem  of  non-unique  access  is  to  select  a 

set  of  keys  so  that  the  probability  (3)  is  near  unity. 

Let  us  first  solve  the  problem  of  calculating  probability  (3)  by  study- 
ing the  case  of  a  single  key,  i.e.,  the  probability 

P(M.|Kj.  &  A)  (4) 

By  the  multiplication  principle  we  have 

P(M. |A)  P(K, |M,  &  A) 

P(M.|K.  &A)=   (5) 

'    J  P(Kj|A) 

For  readability  we  will  omit  the  reference  event  A,  thus  (5)  assumes  the 
simpler  form. 

P(M.)  P(K.1M.) 

p(M  |K  )  =   ^  ^—L-  (6) 

'    J  P(K.) 

The  factors  on  the  right  have  the  following  meaning: 

P(M. )  =  the  a  priori  probability  that  a  (7) 
'       randomly  selected  record  matches 
the  individual  i 


P(M.)  =  1/N  if  N  is  the  total  population  upon  which  the  data  bank  is 
drawn.*    For  example,  if  the  individual  i  is  known  to  be  a  California 
driver  and  the  data  bank  contains  all  and  only  California  driver  records, 
then  N  =  the  size  of  the  data  bank.   On  the  other  hand,  if  the  data  bank 
concerns  information  on  adult  U.S.  citizens,  then  N  =  the  number  of  adult 
U.S.  citizens. 


In  the  complete  formulation  with  the  reference  event  A,  A  could  be  con- 
sidered to  contain  the  information  regarding  the  population  upon  which 
the  data  bank  is  based. 
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We  next  have: 


P(Kj|M^. )  =  the  probability  that  a  record  contains  (8a) 
a  specific  value  for  retrieval  key  j 
conditional  on  the  fact  that  the  record 
matches  the  individual  i. 

For  example,  if  the  j  "  key  is  i's  address,  then  P(K. |M. )  is  the  prob- 
ability that  i's  record  contains  his  correct  address.  It  therefore 
follows  that 

P(Kj|M^.)  =  the  complement  of  the  sum  of  the  error  (8b) 

rate  in  key  j  and  the  relative  frequency 

of  omission  of  data  in  key  j 

Note  that  for  perfectly  accurate  records,  we  would  have  P(K. |M. )  =  1.* 

The  third  factor  P(K.)  is  estimated  as  the  relative  frequency  of  the 
specific  information  given  by  key  j  in  the  bank.    For  example,  if  two 
records  are  retrieved  having  the  same  value  for  key  j  and  m  is  the  size 
of  the  data  bank  then  P(K.)  is  estimated  as  2/m. 

Now  if  a  set  of  keys  j  =  1,  n  are  independent,  the  probability  (3) 

is  calculated  by  putting  the  conjunction  K  &  ...  SK^^  for  Kj  in  (5)  (or 
the  abbreviated  Form  (6))  and  expanding  the  result  into  a  product  for 
j  =  1 ,  . . .  ,  n ;  i .  e . , 

n  P(K.|M.) 

P(M.|K,&  ...  &K  )  =  P(M.)    rr         ^  ,  (9) 

J  =  l 

However,  the  independence  assumption  is  only  valid  for  certain  keys.  For 
example,  name  and  birthdate  can  be  regarded  as  independent,  but  not  birth- 
date  and  year  of  graduation  from  high  school.    Independence  also  depends 
on  the  conditional  probability.    We  therefore  must  look  for  a  more  general 
solution. 


The  error  rate  in  query  formulation  is  not  part  of  the  probability  (8)> 
which  has  to  do  with  the  probability  of  the  selection  procedure  being 
successful  conditional  on  an  error-free  query. 
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One  approach  to  this  problem  is  to  use  some  probability  machinery 
developed  by  W.  E.  Johnson  and  reported  in  Keynes. 53  There  are  also 
versions  of  the  Johnson  work  in  Carnap.54 


We  now  present  our  version  of  the  so-called  "Johnson  Cumulative  Formula." 
A  set  of  rel evance  coefficients  or  coefficients  of  influence  is  first 
defined.    For  any  propositions  (events)         . . . ,  K  ,  we  define 


P(KJK^  &  A) 

C(K,,KJA)  =   ^— ^   (10) 

^    ^  P(K^|A) 

C(K^,K2  KJA)  =  C(K^,  K2|A)-C(K^  &        K3,  KjA)  (11) 


It  is  then  easy  to  show  that  C(Kj,  K^|A)  has  the  operator  property* 

P(K^  &  ...  &  KjA)  =  C(K^,  KjA)    jf    P(Kj|A)  (12) 

(see  Carnap55).  ^'^ 

The  coefficient  (11)  is  thus  a  coefficient  of  independence  and  is  unity  if 
the  events  Kp  K^,  are  independent  relative  to  the  event  A. 

The  cumulative  formula  is  obtained  as  follows: 

Let  K  be  the  conjunction        ...  &K^.    Then  by  the  multiplication  principle 


P(M. |A)    P(K|M.  &  A) 

P(M.|k&a)  =  —  (13) 

'  P(K|A) 

Applying  (12)  to  the  numerator  and  denominator  of  (13)  and  then  multiply- 
ing both  sides  of  the  result  by  [P(M. jA)]"-!  we  get 


We  use  our  own  notation  for  the  coefficient. 

53 

Keynes,  J.  M. ,  A  Treatise  on  Probability,  MacMillan  and  Co.,  London,  1929, 
pp.  149-155. 

54 

Carnap,  R. ,  The  Logical  Foundations  of  Probability,  University  of  Chicago 
Press,  1950,  pp.  356-360. 

^^Ibid,  p.  358,  Formula  T66-3C. 
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(14) 


,1  C(K,,  K  JM.  &  A)     n     /p(M..  |  A)  P(K.iM.  &  A) 

[P(M.|A)]"-^  •  P(M.|K  &  A)  =  — i   n  \     ,  '  

'  '  C(K^,  K    A)  j  =  l    \  P(K.lA) 


We  now  use  the  cumulative  formula  (14)  in  the  following  way.  First  intro- 
duce the  abbreviations. 


V  =  P(M. |K  &  A)  (15) 
P  =  P(M. |A)  (16) 


qj  =  P(MilKj  &  A)  (17) 


By  the  multiplication  principle,  q.  is  the  factor  in  the  parentheses  of 
(14),  i.e.,  J 

P(M. |A)  P(K.|M.  &  A) 

q.  =  —  ^   (18) 

^  P(Kj|A) 

Our  problem  is  to  study  the  expression  for  V.    Using  the  abbreviations, 
the  formula  (14)  becomes 

,         C(K,,  K  !M.  &  A)  n 

p"     =  — ^-  ^— !   TT    q,  (19) 


C(K^,  KJA)  j  =  l 


J 


We  now  use  (19)  again,  but  this  time  substitute  '^M. ',  i.e.,  the  negation 
of  M-j ,  for  'Mj'  in  (14).    The  quantities  p,  V,  and       are  then  replaced 
by  their  complements.    We  get. 

n  1  C(Kt,  K  |%M.  &  A)  n 

(1-p)"-^  (1-V)  =   '      \  TT    (1  -  q.)  (20) 

C(K^,  KJA)        j  =  l  J 
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The  ratio  of  (20)  to  (19)  yields 


where  E  is  the  factor 


C(K,,  K\-M.  &  A) 

E  =   i  ^  ^   (22) 

C(K^,  K  |M.  &  A) 


The  equation  (21)  is  the  solution  to  our  problem;  that  is,  the  right  side 
of  (21)  is  the  product  of  the  two  quantities,  E  as  defined  by  (22),  and 

-(Tr)""\rr(^^) 


so  that  (21)  becomes 


from  which  we  derive 


T   -    1  =  EF 


V  =-i 


1  *  EF  (24) 


We  see  that  V  depends  not  only  upon  the  quantities  q.  but  also  on  p,  the 
a  priori  probability  of  M. ,  and  the  quantity  E.  The"^ value  used  for  E  is 
usually  obtained  by  independence  assumptions. 


Note  that  if 

E  <  1 


then,  from  (24), 
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V  >  — 

^         1  +  F 


(25) 


This  means  that  a  lower  limit  to  the  probability  of  M^.  conditional  on  the 
combination       &  Kp  &    ...    &      can  be  computed  if  E  £  1.    The  value 
E  =  1  occurs  when  the  numerator  and  denominator  of  (22)  are  equal.  In 
particular,  this  is  the  case  when  the  keys  are  independent  relative  to 
both  ~M^-  and  M. .    The  smaller  the  value  of  E,  the  greater  the  ability 
of  the  keys  to  discriminate. 

To  summarize:    We  have  three  formulas  for  the  desired  probability  V  each 
involving  a  different  ratio  of  coefficients  of  influence:    the  formula 
(19),  the  formula  (20),  and  the  formula  (24).    In  Section  5  we  have 
implemented  the  formula  (24)  for  a  range  of  values  of  E. 
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APPENDIX  III 


SOFT-MATCH  TECHNIQUES 


In  Section  5.2,  we  noted  the  classical  problem  of  trade-off  between  re- 
call and  precision,  i.e.,  increasing  recall  lowers  precision.    But  this 
arises  only  in  the  case  where  all  retrieved  records  are  treated  on  an 
equal  par  and  not  subjected  to  further  filtering.    Our  approach  to  the 
problem  is  to  increase  the  recall  and  then  to  provide  a  mechanism  for 
additional  filtering  to  improve  the  precision.    In  this  section  we  dis- 
cuss a  method  for  increasing  the  recall  called  "Soft-Matching."  The 
result  of  a  Soft-Match  Technique  is  not  only  a  set  of  retrieved  records, 
but  also  a  numerical  weight  assigned  to  each  record  to  indicate  its 
closeness  to  the  query.    As  an  example,  consider  the  case  of  a  query 
which  is  a  conjunction  of  three  key  values  denoted  by  A,  B,  C.    A  tradi- 
tional retrieval  would  select  only  records  that  matched  on  all  three.  A 
soft-match  technique  might  be  to  also  select  those  records  that  matched 
on  A  and  B  but  not  on  C,  on  A  and  C  but  not  on  B,  and  on  B  and  C  but  not 
on  A.    These  secondary  retrievals  would  then  be  assigned  weights  less 
than  that  assigned  to  a  record  that  matched  on  all  three.    The  retrieval 
could  also  be  extended  to  tertiary  retrievals  such  as  records  that  match- 
ed on  A  but  not  on  B  and  not  on  C,  etc.    These  would  be  given  the  lowest 
weights. 

In  this  appendix  a  heuristic  weighing  scheme  is  presented  for  filtering 
the  candidate  records.    In  Section  5.5  and  Appendix  II  we  presented  a 
second  weighting  scheme  based  on  probabilities  which  is  in  fact  the 
measure  of  precision  for  a  single  retrieval.    (That  is  to  say,  precision 
IS  the  proportion  of  retrieved  documents  that  are  relevent,  i.e.,  match 
the  inquirer.    Consequently,  if  exactly  one  record  is  retrieved,  then  the 
precision  corresponds  exactly  to  the  probability  that  the  record  is  in 
fact  that  desired  by  the  inquirer.) 

On  the  basis  of  the  above  discussion,  we  see  that  a  soft-match  technique 
involves  two  components:    (1)    an  algorithm  for  determining  a  subset  of 
the  data  base  of  manageable  size  which  has  maximum  recall;  (2)    a  scoring 
function  for  assigning  the  closeness  measures  to  each  item  in  the  subset. 

First,  consider  the  problem  of  determining  a  manageable  subset.  Suppose 
that  the  retrieval  prescription  involves  n  keys 


Then  let  S(K. )  be  the  set  of  records  which  match  on  the  i^"  key.  In 

principle,  the  subset  with  maximum  recall  would  be  the  set  union  of  S(K^), 

S(K^).    This  approach  would  not  be  efficient  however,  so  a  method 

which  utilizes  the  error  and  omission  rates  and  the  discrimination  factors 
is  recommended.    These  quantities  are  defined  as  follows: 
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a^.  =  probability  that  the  key  value  for  the  i     key  is  incorrect 

b^.  =  probability  that  the  key  value  for  the  i^*^  key  is  omitted 

e.  =  a.  +  b. 
1       1  1 

(Thus,  the  complement  of  e^.  is  the  quantity  c^.  defined  in  Section  5.1). 

The  discrimination  factor  of  a  key  is  measured  by  the  relative  frequency 
of  a  key  value  in  the  data  base.    Let  R^.  be  the  relative  frequency  for 

the  i^^  key.    This  quantity      is  estimated  for  certain  keys  by  the 

values  given  in  Tables  5.3-1,  5.3-2;  it  therefore  corresponds  to  the 
concept  of  retrieval  ratio. 

It  follows  that  the  initial  selection  of  the  retrieval  should  be  the 
union  of,  not  all  n  sets  S(K^. ),  but  a  certain  selection  of  fewer  sets 

say  S(K.  ),         S(K.  )  (m<n),  that  has  the  properties 
h  ^m 

(1)  R.    X  R.    X  ...  X  R.    is  less  than  a  certain  threshold  r  . 

'l       '2  \  ° 

(2)  e.    X  e.    X  ...  X  e.    is  less  than  a  certain  threshold  e  . 

h       '2  \  ° 

The  first  property  guarantees  that  the  retrieval  set  has  manageable  size. 
The  second  property  guarantees  that  the  probability  of  missing  a  relevant 
record  is  low. 

The  second  aspect  of  the  soft-match  technique  requires  the  definition  of 
the  scoring  function.    For  this  purpose  consider  that  we  have  an  arbitrary 
quantitative  "strength  of  match"  function  for  each  key.    For  example,  for 
a  key  designating  home  address,  one  such  function  might  be 

f^.  =  1      for  complete  match 

f.  =  .5     for  agreement  on  the  street  name,  part  of  the  home 
address  but  not  on  the  house  number  part 

f.  =  0  otherwise. 
1 

More  complicated  functions  can  be  defined  in  obvious  ways.    For  example, 
if  the  house  number  partially  matched,  say  it  disagreed  on  one  numeral 
out  of  four,  then  we  could  score  this  as  .75.    The  simplest  function  is 
the  binary  function,  '1'  for  match,  '0'  for  no  match. 

In  general,  we  see  that  a  score  for  a  key  for  which  f^-  >  0  (i.e.,  a 
partially  matching  key)  should: 
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(1)  Increase  as  R^.  decreases 

(2)  Increase  as  f^.  increases 


(3)    Increase  as  a^  (the  error  rate  in  the  i^*^  key)  decreases. 

It  is  therefore  plausible  to  take  the  contribution  of  the  i^*^  key  to  the 
total  score  (in  the  case  that  f^.  >  0)  to  be  proportional  to  the  quantity 

f .  (1  -  a.) 


R. 

since  this  is  a  simple  expression  that  satisfies  the  requirements  (1), 
(2),  and  (3). 

It  is  convenient  to  normalize  the  scores,  however,  so  we  will  take  the 
constant  of  proportionality  to  be  1/R  where 

"  1 

R  =  2      1-  (1) 
i=l 

J.  L. 

Thus,  the  score  for  the  i     key  is 

We  now  define  the  total  score  as  consisting  of  two  components:    a  positive 
component  made  up  of  the  contributions  S.  defined  above  and  a  negative 
component  to  be  defined  below.  ^ 

The  positive  component  we  take  to  be  simply 

n 

(Note  that  this  summation  runs  over  all  the  keys  in  the  query,  and  not 
over  just  the  subset  of  m  keys  used  to  get  maximum  recall.) 

By  (1)  and  (2)  we  see  that  S  is  a  convex  linear  combination  of  the  terms 
f^.  (1-a^. )  where  i  =  1,  n,  and  so  S  can  range  from  0  to  1. 

To  define  the  negative  component,  we  see  that  if  a  key  is  error  free  and 
fails  to  match  (f^.  =  0),  then  it  should  kill  off  the  entire  retrieval. 

Thus,  in  the  case  f^.  =  0  we  should  have  a  negative  contribution  which 
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varies  as  (1  -  a.).    We  therefore  take  the  score  when  f.  =  0  to  be 
1  1 

=-(l-a.) 

Since  a  value  of  T^.=  -1  fbr  only  one  key  is  sufficient  to  invalidate  a 

retrieval,  we  take  the  total  negative  score  to  be,  not  the  sum  of  the 
individual  scores,  but  rather  the  negative  of  the  maximum  value  of 
(1  -  a. ),  i .e. , 

T  =  -  max  (1  -  a . ) 

f  .=0 

Thus  T  ranges  from  -1  to  0. 
The  retrieval  scoring  function  is  thus 

U  =  S  +  T 

An  example  of  how  this  scoring  function  is  used  is  given  in  Table  III- 
The  example  deals  with  the  scoring  of  four  records  based  on  values  of 
the  parameters  R.  (the  retrieval  ratio)  and  a.  (the  error  rate). 
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Table  III-l.    Example  of  Soft-Match 


KEY 

NAME 

ADDRESS 

SEX 

.003 

.00002 

.5 

^• 

.001 

.01 

0 

Table  of  Values  for  R^. ,  a^. 


NAME 

ADDRESS 

SEX 

Record  1 

match  (1) 

match  (1) 

no  match  (o) 

Record  2 

match  (1) 

partial  match  (.75)* 

match  (1) 

Record  3 

partial  match  (.83)** 

match  (1) 

match  (1) 

Record  4 

match  (1) 

match  (1) 

data  absent 

Table  of  Retrieval  Outcomes 

(values  in  parentheses  are  those  for  f. ) 


S 

T 

U 

Record  1 

.9900 

-1 

-.0100 

Record  2 

.7442 

0 

.7442 

Record  3 

.9889 

0 

.9889 

Record  4 

.9900 

0 

.9900 

Retrieval  Scores 


*  The  partial  match  example  is  for  the  case  that  one  character 
out  of  four  doesn't  match. 
**  The  partial  match  example  is  for  the  case  that  one  character 
out  of  six  doesn't  match. 
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APPENDIX  IV 

FORTRAN  SOURCE  CODE  FOR  THE  PROBABILITY  MODEL 
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FORTRflH  IV 


!>06-^ll     SOURCE  LISTIHG 


PRGE  001 


C  >»(5(<M")<;?(<*>«>K-»t>(o)<'t"»<*'t"<«*>tt*'.Jt< 

C  QUERY  SIMLtLATES  THE  Di^TA  BASE/USER  INTERhCTIOH 

9081  DIhEHSTOH     <  1  H . B (  1  D , C H  D , R C  1  1) , S (  1  1 ) 

0002  DIMENSION  JARV;  1  H  ,  IFLD  ( 11  > 

0003  D  I  MENS  I  OH  PiRGS  (1  H 

0004  DIMENSION  SURHM( 10) 

0005  COMfiuH   IRAN,   JRAN-   EARY(2?;  ,  PARV<3:i 

0005  DATA  ARCS  <.  1 )    .  239726E- 2^,  ARGS  (2)  /  .  833333E-       ARGS<35  /  .  02/' 
000?  DATA  ARGS  (4)  /  .  02-^,  PRGS  (5)  ^.5-^,  ARGSJiG )  /  .  600 1^',  ARGS  (7) 

0006  DATA  fiRGS  (  6)  /  .  66b6666/",  ARGS  (9)  --'.000333333/,  ARGS  (  10)  / .  555555E-4-' 
0009  DATA  SUR'-IM(1)..SURHM(2)  -SURNM(3)  ,SURNMC4)/.9e7E-?,  .2459E-5,  .9S35E- 

2  ,  .272037E--4/ 

00)0  DATA  SURHM(5)  ,SURNM(6)  ,SURHM(;7)/4.81966E-5,6.627E -5..  1.265E-4.'' 

00  1  1  DATA  SURNMC8)  ,  SURHM<9)  ,  SURNMC  t  Q)  /4 . 6  1 97E-4,  7  .  S32E-4..  .  3384E-2/ 

0012  DATA              ,  PARVf  1)  ,PAPY<2)  ,PARYC3)..-'4.545454E-9,2.5E-7,  1  .E-6/-- 

00  13  DATA  EARY  (  1 )  ,  EAR Y<  2 )  ,  EAR Y ( 3 )  ,  EARY  ( 4)    .  5 ,  1  .  ,  2  .  ,  1 0  ,  / 

c 

C  QUERY  SIMULATES  QUERY  PROCESSING  AGAINST  A  DA  I A  BASE. 

C  IT  GENERATES  VALUES  FOR  A (J) L THE  ERROR  RATE   IN  A  FIELD] 

C  B(J)     [THE  OMISSION  RATE  OF  DATA   IN  A  FIELD] 

C  C  ( J  )     C  1  -  ( A  (■  J )  +B  ( J )  ]  .    .  THE  COHF  I DENCE  FACTOR 

C  R(J)     THE  PERCENTAGE  OF  THE  FILE  RETRIEVED  VIA  A  FIELD,  AND 

C  Si; J)     A  QUERY  SCORE,   IJHICH  RANGES  FROM  1  TO  1"HE  FILE  SIZE. 

C  SU)      IS  NOMINALLY  THE  RATIO  CCD/RCJ),   AND   IS  ADJUSTED  HERE 

C  TO  WINTAIN  REASONABLE  LIMITS. 

C  THE  USER  ENTERS   THE  NUMBER  OF  ARGUMENTS   (FROM  1  TO  11) 

C  UHICH  HE  LUSHES  TO  USE   IN  HIS  QUERY.     THE  QUERY  IS  TREftl"ED  AS  IF 

C  IT  REQUIRED  A  SUCCESSFUL  BOOLEAN  AND  OF  ALL  THE  ARGUMENTS  TO  BE 

C  SUCCESSFUL. 

C  THE  PROGRAM  LULL  TERMINATE  LJHEN  THE  NUMBER  OF   INPUT  ARGUMENTS 

C  IS  SET  TO  ZERO. 

C 

C  FOP  THIS  TEST  THE  VALUES  OF  A  (J)   AND  B(.J)   bJILL  BE  TREATED  AS  ZERO. 

C  FOLLOUING  THE  ENTRY  OF  THE  FIRST  NUMBER— THE  NUMBER  OF  ARGUMENTS 

C  TO  FOLLOLt--THE  USER  MAY  ENTER  NUMBERS  CORRESPONDING  TO  THE  ACTUAL 

C  ARGUMENTS.     EACH  ARGUMENT  HAS  AN  ASSOCIATED  RCJ),   UHICH   IS  THE 

C  PERCENTAGE  RETRIEVAL  TO  BE  EXPECTED.     THE  ARGUMENT  NUMBERS  AMD 

C  THEIR  ASSOCIATED  VALUES  ARE  AS  FOLLOUS: 


C 


C 

c 

ARG 

UMENT 

NUMBER  MEANING 

RETRIEVAL 

c 

1 

DATE  OF  BIRTH  <MMDD) 

1/365 

c 

2 

MONTH  OF  BIRTH 

1/12 

c 

-3, 

YEAR  OF  BIRTH 

1/50 

c 

4 

STATE 

1/50 

c 

5 

SEX 

1/2 

c 

6 

STREET  NUMBER 

1/ 10, 000 

c 

7 

STREET  NAME 

1/5, 000 

c 

8 

EDUCATION 

1/15 

c 

3 

COUNTY 

1/3000  (ESTIMATED) 

L 

10 

DATE  OF  BIRTH 

(YMD)  i/iS,8yo 

RATIO 


C       BEGIN  BY  ASKING  FOR  OPriRATOR  INPUT. 
0014     1  JRITE(G,205) 

C  5»o|'.'i<**'»"»5'|o|<>H>M:*5|o»^-HoK--H>)<>t< 

C       NOW  EXAMINE  HARGS  TO  START  THINGS  GOING. 
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KORTRftN  IV 


D0b-01     SOURCE  LISTING 


PAGE  002 


6015 
66  ib 


66  J  ?' 
68  1 8 
6019 
8028 
0821 
0022 
0023 
QQ24 
0025 
0026 
0027 


0628 
0629 

0030 

607  1 
6032 
0633 
6634 
0035 
6836 

003? 

6038 
0039 
8049 

0041 

0642 
6043 
8044 
0645 
6046 
604? 
0048 
0049 
6050 
6051 
6052 
0053 
8054 
6055 
085f 
605? 
0058 
0059 

0060 

8061 
0062 
0063 
8064 
6055 
0066 


REAP   (6.216)  HrRGS 
iF-    fHHKGS)    100.  100. 

c     lcme  here  to  prepare  H 


10 


QUERY  SURROGhTE , 


10 


I  =  i 


NhRGS 

(6, 220) 
(6.210)  J 


:i 


32 


33 


34 


35 


3  b 


J,B(J) , J,C(J) , J,R(Ji - J,S(J) , J.JnRVtJ) 


-■R(NARGS-i-l) 


DO  26 
UR  i  TE 
READ 
A  (  I ) =0  . 
B ( I ) . 
C  (  I) =  1  . 
R(I)=hRGS(J) 
S(  I)  =''C(  I)/R(  I)  ) 
JARY<I)=.J 
COHTIriUE 
bJRITt   (6..  2  15) 

C       HOU  CALL  ON  THE  MAIN  COMPUTATIOHAL  ROUTINE  TO  PREPARE  THE  TABLES. 

r;^;1'-;K.-+<-4<>)<-+:>t<?i<>KH«^=1<>H;h:?t<>f<>f"|5* 

JARY(HARG3+1) =1 i 
N=HARC^+i 

yRITE(5,200)  NhRGS 
DO  25  J=l,.  HARGS 
URITE(5,245)    J. A (J) 
25  COHTIi-iUE 

DO  56    I  =  i.  10 
C  (NARGS+i    =  1  . 
B (NARGS+i ) =0 . 
A  (HHRG3+1 )  '-3  .. 
R<NARGS+1> =SURNM( I) 
S(NARGS+1)  =C(NARGS  (-1). 
URITE(5,250) 
DO  45  .J=1..NARGS 
GOTO  <31..32..33.34..35,3b,3?. 
UR^TE(5,255) 
GO  I  0  4b 
WRITE (5- 260) 
GOTO  45 
UR^TE(5,.265) 
GO  I  0  4b 
t.JPITE(5,2?e) 
GOTO  ^45 
LJRITE(5,2?5) 
GOTO  45 
LiR  ITE(5  .280) 
GOTO  45 
UR  RE  (5..  285) 
GOTO  45 
UP  I  TE  (5..  290) 
GOTO  45 

39  UJR_I_TE(5,295) 
GU!0  45 

40  LJRITE(5,3e0) 
45  CGHTIHUE 

WRITERS, 305) 
yR!TE(5, 150) 
URITE    (5,225)  I 
yRI-E'5.23G) 


30,39,40^  JARYtJJ 
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FORTRCIN  IV 


D0b-01     SOURCE  LISTING- 


PAGE  003 


6068 
0069 

00?0 
00?  1 
00?2 

00?3 


00|-'4 

150 

200 

00?6 

2  0  'J 

00?? 

2  i  '3 

00  ?l.i 

2  1  b 

0  0  ?  9 

220 

0  0  y  0 

003  1 

230 



u  y  ti 

 ^ 

2  J  'J 

00S3 

246 

0064 

245 

0035 

250 

0086 

255 

008? 

26? 

0088 

265 

0089 

2?0 

0036 

2r5 

009  i 

260 

0092 
0393 

285 
290 

0094 

2'='5 

0095 

300 

0096 

365 

009? 

13X,  1  10('-' ) 
'HhRG3  =  '  ..  12.) 
ENTER  NUMBER 


iJRITE':5,235:j 
I  I  h  i;5,  240) 

CALL  pycY';H,e,c, R.. 

30  CONTINUE 

GO  TO  i 
1 1?  0       C  »Ll.  P  VC  V  ( H ,  B ,  C  ,  R .. 

CALL  EXIT 

C       FORMAT  STATEMENTS  - 

FORMAT 
FORMAT 
FORMAT 
FORMAT 
FORMAT 
FORMAT  r 
FORMAT' i H 
I    '■  SURNAME 
FORMAT  f,  IH 

1  IH   ,  i3X.. 

2  2X^    !  '  ■  23X 
FuRr:AT(  i  H   .  13X,  '  !  '  .. 

1  If!   ,  13X,  ■■  !    ,  15X,  '  ! 

2  '  4  MILL  ION''  ..  8X,  '  !  '" 
FORMAT'.:  IH  ,  13K,  '  !  " 
FORMAT':  1H0  ■  '  A  ('  .  12,  ' 

1   3X.  'C(  -  ,  12,  '  )  ='  ,E15 


N, JARY) 


B,NARGS, JhrY) 


?X> 

OF  ARGUMENTS 


IH  , 
i  H  i , 
( '  0 
(12) 

!•; '    EXITING  QUERY'  ) 

r   ENTER  QUERY  FIELD  IDENTIFIER') 
13X. •  r , leeX, ' ! '.?K/1H  ,13X, ' 


.  43X, 


DEUlLt '  ,  IX,  12,48,><,,  '  !  ' 
,  13X,  '  !    ,  108('-'  )  ,  '  !  ' 
! ' , 3X, 'ERROR  RATE ' , 2X 
ILE  SIZE' ,3?X, ' ! ' 
,  '  !  '  ,  ibX,  '  i  '  i?X, 


'^1 
,  5X, 
108  ( 


?X 


1 


dX, ' RhCALL  ^ ACTOR' , 
,  10X, 


220  MILLIUH' 

E15.8,3X,  'B( 
i;X,  'RC  ,  12,  '  > 


f4( ' -' > 
MILLION 
bX.  '  ! 


'XX 
t  ■■ 


2   12,    )  =  '  .E15.8,3X,   JARYC  ,  12.-  '  )  =  '  ,  12) 
FORMAT  a  H 1 ,  43X, ' RREC I S ION  OF  RETR lEVED 
1  ,1-1  ELDS') 


FORMATC IH 
FORMAT ( IH 
FORMAT';  IH 
FORMA^;^  IH 
I- UK  MA  !  (  IH 
FORMATC IH 
FORMAT ( IH 
FORMAT';  iH 
FORMATC IH 
FORMiATC  IH 
FORMATC IH 
END 


,  48  X. 
,  46  X 
,  48X. 
,  48  X. 
,48X^ 
,  48  X. 
,  48 X. 
,  48X. 
,  48  X, 
,  4SX. 
,  48  X 


12, ' ) =' ,Ei5.8, 
,E15.8,3X., 'SC 

RESULTS  FOR  DATA 

) 


'DATE  OF_BIRTH   (MONTH  AND  DAY  ONLY) 
'MONTH  OF  BIRTH' ) 
' YEAR  OF  B IRTH' ) 
'STATE') 


STREET  NUMBER' 
STREET  NAME' ) 
EDUCATION' ) 
'  COUNTY' ) 
'DATE  OF  BIRTH 
'AND  SURNAME') 


(YtHR,   MOH'ri,   AND  DAY)' 
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STORAGE  MRP 


H 
El 
C 
R 

S 

JARY 

IFLD 

HRGS 

SURHM 

HARGS 

I 

J 

N 

PVCY 
EXIT 


606006 
005062 

6130212 
I3002G6 
9063.42 

0005P0 

600416 
000472 

002100 

002102 
602104 
002 106 

000000 
000000 


COMMGN  BLOCK  // 


IRAN 
JRAN 
EARV 
PAR't' 


000000 
000002 

808004 

000160 


REAL*4 

REAL.-K4 

REAL*4 

REAL.--h:4 

REAL*4 

1NTEGE:R'.k2 

IHTEGER*2 

REAL +4 

REAL?K.4 

IHTEGER't<2 

INTEGER*2 

Ih  rEGER=i-2 

il-iTEGER>i<2 

REAL>f:4 

RErtL*4 


ARRAY 
ARRAY 
ARRAY 
ARRAY 
ARRAY 
ARRAY 
ARRAY 
ARRAY 
ARRAY 
VARIABLE 
VARIABLE 
VARIABLE 
VARIABLE 
PROCEDURE 
PROCEDURE 


(11) 
(11) 
(  1  1) 
(11) 
(  1  1  ) 
(  1  1 
(  1  1 
(  1  1 

(10 


LtNGTH  000174 


IHTEGERf^S 
IHTEGER*2 
REhL:-k-4 
REAL-+:4 


VARIABLE 
VARIABLE 
ARRAY  (27) 
ARRAY  (3) 
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FORTRAN  r 


D06-01     SOURCE  LISTING 


PhGE  001 


Sua  i 


6002 
f=)B03 
0004 
13065 

0006 
000? 


0008 


''1009 
06  10 

06  1  i 

00  12 
00  1  3 
00  14 
0015 
00  15 
0M  t  ? 
0016 

09  19 


0020 
003  i 
0022 
0623 
602';! 
Q025 


W  L1 2  r3 
''■'32? 
0029 
0030 
003  1 
0032 
9033 


bU34 
0035 


PRIVACY  .MCDEL--MARi::H  19?6 
BEEN  MODIFIED  TO  PRODUCE  H 


C.^'■.>t'^+^?K^K>K'1<'^<.H:>^^^^?K^^<'t'**>^>^<>K?K 

C  liHlTl  COMPUTATION  ROUTINE  OF 

C  ^»<|--iOTe:    THIS  SUBROUTINE  HAS 

(J:1<j|<* ■■■KJi<>*<H<'lc-trH;>t;>h;<<->K>|<>K>t'5t< 

SubKUi  JTIiiE  PvT:y(  A  J  .  BJ,  CJ,  RJ,  SJ,  NARGS,  JARY) 

C  ^~>KH■^K*>l<^'>t•M<*.'t■  ?h-i<.'K-+;K*;(<>l<.+: 

C  DECLARE  VARIABLES 

C;t^^^<;K-+-*H^5t^;^•>KH<:H■"l■*>^:■■l^>l<>K>^<>K>^< 

D  IMEHS  luhi  jhRy(  1  i ) 
DOUBLE  PRECISION  B 
COMMON 
REAL  A 


TABULAR  PRINTOUT. 


:GPH  1  n  .F.VARYn3,3J 
I  RAN ,  JRAH ..  EAR  Y(  27 ')  ,  PARV  ( 3 ) 
J  i  1  1 )  ,  BJ  <  1  1 .)  ,  C J  (■  1  1)  ,  RJ  (11)  ,  S J  (in 


KtP'.  uARi'(ir) 

DATA  VARY  (  1 ,  1 ) . ''9  .  ODO--' 

C  MAIN  LOOP  STARTS  HERE 
C 

C  VALUES  ARE  RETURNED   IN  THE  ARGUMENT  ARRAY: 

IF  (NARGS)    95..  95^  1  1 

C  BEGIN  PROCESSING  THE  QUERY  RESPONSE  HERE 

1 1  CONTINUE 

DO  90  JK  ■=  1,13 
L=.TK 

CONF  = 1 . -0 . 005* ( L- 1 > 
R  E  F  A  C  =  C  0  N  r      ( N  A  R  G  S ) 
ERRAT= 1 . -CONF 
PERERR=ERRAT>t^i00 
DO  80   I  =^  i ,  3 
P=PhRY( I) 
DO  20   II  =  i , HARGS 
20         QARY  (  I  T )  =pJt<S  J  (  I  I )  *CONF 

C-^***^'**H^-'+^?^:^->t"t<?i<:^^'.io|'>t<>K 

C  NO(..J  COMPUTE  THE  FUNCTION  F 
F=l  ., 

K  =  P.^f  1  .  -P) 

DO  30  J.J=1  -NARGS 

B  I GP  I  ( J  J )  -  (  1  .  -  Q  A  R  Y  ( J  J )  )  .^3  A  R  Y  ( J  J ) 
30         F«F*X*B IGPI ( JJ) 
F=F..-<K 

C  PREPARE  THE  PFIHTOUT  TA^LE 

VARY  ( Ji< ..  I'l  =  1  .  .'■  ( 1  .  +F ) 

.GT 


-TERMINATE   IF  NARGS- 


IF(VARY(.JK,  t; 
CONTINUE 
LIRITE';5,  150) 
CON  I iNUE 

1  ?  0 ) 


1.0)  )   VARY(JK:,  D-l  .0 


PERERR,REFAC,  ^VARYcJK,  V)  ,  i  - 


1,-1) 


IJRITEC; 
RETURN 

C  E X I  r  V  J  A  HERE  UH EN  W I- :R G S  =  0 

95         bJR  I  TE  (5,  200) 
RETURN 
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FGR"^RfiN  IV 


1)06-01     SOURCE  L  I  STING 


PfiGE  002 


C  F    'MAT  SThTEMEHT  SECTION 

r"  ^^;^■l'.-^<;^^,^:■^■•.;^■^^Cil■c.■!^^{^f^;^K;|^?K^K■^<^K>l< 

ei33G     156      FORMAT  <1H  ,  13X, '  '  \' ,  17X,  *  ! ' ,  23X- '  1' ,  2?X,  '  f',22K, ' 

1  IH  ..  13X,.  '  r  ..4X,FS.2,       ,4X,.  '  !  '  ,4X,FS.4,5X,  ■=  !  %4X.. 

2  F  i5.  8,4>^,  '  i  %6X.E15.8,6X,  '  I  '  ,  3X..  E  15  .  3,4X,  '  ! '  ,7X) 
0037     170       PORMATCIH  ,  i3X,  ilQC  "'),7X) 

eB'^S     Z-tm       FOPhMTdHG,'         HORMRL  TERMIHRTIOH' ) 
003:?  EhiD 
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FORTRfl!^  IV 


STORflGE  MhP 


IHTEGFRf<2 

PhRhMETER 

ARRAY  (11) 

B  IGP  I 

REi-lL--i;8 

flRRHN'  <11) 

VhRy 

f?i!3C'  162 

REHL''Cd 

ARRAY  (13- 

5,1  VElTOReI; 

A,  J 

REAL*4 

PARAMETER 

ARRAY  (11) 

B.J 

6660  lb 

REHL-i^4 

PARAhETER 

ARRAY  ( 1 1 ) 

CJ 

666^26 

REHL>h:4 

PARAMETER 

ARRAY  (11) 

R  J 

QQtiy22 

REHL+-4 

PHKHllETbR 

ARRAY  (11) 

G.T 

Pl;'6e24 

RErtL*4 

PARAMETER 

ARRAY  ( 1 1 ) 

QhRV 

01^0652 

REhL*4 

ARRA'r  (11) 

T  L  1  "T*       i~'  f"  l™i  .1^  —1 

I!iTciiER*.2 

PmPH'IETER 

VriR IhBLE 

F 

60  I i  46 

REAL-N8 

'v'AR  I  At'iLE 

P 

06  1  i  50 

R£hL>k4 

VhR  IhBLE 

JK 

00  1  154 

1  N  1  Ebl.;R?)<2 

vhR  iHbLE 

L 

06  1  i5b 

INTEGER  ••(^2 

VARIABLE 

CONF 

661160 

REhL^4 

VARIAeLE 

REFAC 

i^Sl  164 

REAL>k4 

VARIABLE 

E^RRhT 

061  i?6 

REf-iLn<4 

VARIAPi.,E 

/ 

r"i  r~"  r*-!  r~  r~i  i—i 

perekr 

00  i 1 74 

REhL>k4 

VARIABLE 

I 

00  1  200 

IHTEGERK2 

VARIABLE 

I  I 

06  1 202 

IHTEGER'tsZ' 

VARIABLE 

X 

06  1264 

rehlm 

VARIABLE 

J  J 

001210 

iNTEGERt<2 

VARIABLE 

CUfiMfiN 

tlL:Ji:K  ..-v 

LENGTH  000174 

trhh 

000060 

INTEGER'^2 

VARIRPLE 

jrhh 

000662 

IHTEGERi^2 

VARIABLE 

Ef'iRY 

Pian064 

RE!^L*4 

ARRAY  (2?) 

PfiRY 

000 160 

REi-iL>i':4 

ARRAY  (3) 
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